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CHAPTER  1 


IWi'RODlJCTION 


1.1.  Overview  and  Thesia  Organization. 

In  the  past  decade,  several  constructive  proofs  of  the  Brouwer 
and  .Kakutani  fixed  point  theorems  have  emerged.  These  proofs  have  been 
developed  into  algorithms  (known  in  the  literature  as  complementary 
pivot  algorithms;  which  search  for  fixed  points  on  unbounded  regions. 

In  turn  these  algoritlims  have  been  used  to  solve  problems  arising  in 
economics,  engineering  and  other  branches  of  applied  mathematics.  An 
important  application  for  which  this  method  was  awkward  was  that  of 
optimizing  an  objective  function  subject  to  both  equality  and  inequality 
constraints  (hereafter  referred  to  as  the  general  constrained  optimi- 
zation problem).  One  result  of  the  dissertation  is  the  most  efficient 
complementary  pivot  algorithm  to  dale  for  handling  this  problem.  The 
second  major  contribution  of  tids  thesis  is  a general  structure  on 
fixed  point  problems  which,  when  present,  enables  one  to  work  in  a 
lower  dimensional  space.  It  is  shown  that  the  general  constrained 
optimization  problem  may  someti.mes  be  formulated  as  a fixed  point 
problem  possessing  this  property. 

The  basic  approach  adopted  in  this  work  for  handling  the  general 
constrained  optimization  problem  is  to  use  an  implicit  function  (derived 
from  the  equality  constraints'  to  solve  for  some  depemlent  variables  in 
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terms  of  the  remaining  independent  ones.  Under  certain  circumstances, 
a fixed  point  algorithm  may  be  used  to  search  for  optimal  values  of  the 
independent  variables  while  Newton's  method  for  solving  nonlinear  systems 
of  equations  is  used  to  determine  values  of  the  dependent  variables. 
Theoretical  conditions  on  the  original  functions  are  developed  to 
guarantee  that  the  fixed  point  algorithm  converges  to  a solution  and 
various  techniques  are  devised  to  enhance  the  overall  efficiency. 

To  help  ascertain  the  value  of  this  method,  comparative  computer 
tests  are  run  against  the  Generalized  Reduced  Gradient  (GRG)  algorithm 
^ich  is  a well  established  nonlinear  prograunming  code.  This  method  was 
selected  as  the  basis  for  comparison  because,  to  the  author' s knowledge, 
it  is  the  best  commercial  code  for  solving  the  general  constrained 
optimization  problem  (see  Colville  [81,  Nishiyama,  Simkin  and  Takeuchi 
[to],  Lasdon,  Warren,  Jain  and  Ratner  [3^]).  Seventeen  test  problems 
were  taken  from  various  sources.  The  fixed  point  code  solved  all  seventeen 
and  GRG  solved  sixteen.  This  supports  the  robustness  of  the  fixed  point 
approach.  As  to  the  cort^puter  times,  the  fixed  point  code  proved  to  be 
as  fast  or  faster  than  GRG  on  the  lower  dimensional  problems.  As  the 
dimension  increased,  however,  the  trend  reversed  and  on  a forty  dimensional 
problem  GRG  was  approximately  eleven  times  faster.  The  conclusion  is 
that  when  the  dimension  of  the  original  problem  can  be  sufficiently  reduced 
by  the  equality  constraints,  the  fixed  point  approach  appears  to  be  more 
effective. 


2 


The  dissertation  consists  of  seven  chapters.  Chapter  2 contains 
the  essence  of  the  dimension  reducing  property  along  with  several  examples 
of  where  this  structure  arises  in  applications  (of  principal  interest  is 
the  general  constrained  oDti'^i.ia.Lion  proDiem) . Under  certain  circumstances, 
the  algorithm  described  in  Chapter  'j  can  be  used  to  obtain  a solution  to 
this  problem.  Some  theoretical  conditions  on  the  original  functions  which 
ensures  that  the  algorithm  converges  to  a solution  are  established  in 
Chapter  4 while  Chapter  5 deals  with  all  of  the  computational  consider- 
ations. Chapter  6 proposes  various  techniques  to  improve  the  efficiency. 
Finally,  Chapter  7 presents  the  results  of  the  computer  tests  along  with 
the  appropriate  conclusions. 


1.2.  Historical  Development. 

The  history  of  the  computation  of  fixed  points  dates  back  to  1929 
when  Knaster,  Kuratowski  and  Mazurkiewicz  [30]  gave  the  first  constructive 
proof  of  the  Brouwer  fixed  point  theorem  using  Sperner' s lemma.  It  was 
not  until  thirty  eight  years  later  that  Scarf  [i+8],  using  the  ideas  of 
complementary  pivot  theory  developed  by  Letnke  [33]  and  Lemke  and 
Howson  [36]  in  196^,  produced  the  first  algorithm  to  approximate  fixed 
points  of  continuous  functions  from  the  simplex  into  itself.  Cohen  [?] 
simultaneously  developed  a constructive  type  proof  of  Sperner 's  lemma. 

Many  ideas  closely  related  to  fixed  point  computation  were  anticipated 
by  Hirsch  [27]  in  1963  wherein  he  gave  an  existence  proof  by  using  a 
certain  constructive  technique  to  reach  a contradiction. 
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The  combinatorial  techniques  of  Scarf  s algorithm  have  various 
other  applications  in  mathematics  and  economics  as  shown  in  Scarf  and 


Hansen  ^25].  One  example  of  this  is  approximating  solutions  to  convex 
minimization  problems  in  which  the  feasible  region  is  compact  and  non- 
empty. Another  example  is  the  special  case  of  Kakutani's  fixed  point 
theorem  [28]  in  which  the  compact  convex  subset  involved  is  a simplex. 

There  were,  however,  several  deficiencies  with  these  methods. 
Theoretically,  the  general  version  of  Kakutani's  theorem  could  not  be 
proved  and  computationally  there  was  no  way  to  " continuously"  obtain 
more  accurate  approximations.  Furthermore,  in  higher  dimensions,  these 
algorithms  were  highly  inefficient. 

In  1970,  Eaves  [11,12]  developed  an  algorithm  for  computing  a 
fixed  point  of  the  point  to  set  map  in  Kakutani's  theorem.  Then  in  1971> 
Eaves  [15]  and  Eaves  and  Saigal  [15]  and,  independently,  Merrill  [58] 
developed  techniques  for  overcoming  many  of  the  computational  difficulties. 
The  computer  results  of  these  new  ideas  are  reported  in  Merrill  [38], 
Saigal,  Solow  and  Woolsey  [U7],  Gochet,  Loute  and  Solow  [25],  Wilmuth 
[55]  and  subsequently  elsewhere.  Since  1973  various  other  researchers 
have  contributed  to  the  field  including  Todd  [51,52],  Kuhn  [32,53], 

Garcia  [21,22],  Fisher  and  Gould  [17],  Gould  and  Tolle  [2k],  Kojima  [31], 
Engles  [16],  Friedenfelds  [19],  Eaves  [lU],  Saigal  [kk,k^,k6]  and 
Kellogg,  Li  and  York  [29].  Currently  there  are  over  one  hundred  papers 
relating  to  fixed  point  computation  and  complementary  pivot  theory, 
and  an  extensive  bibliography  may  be  found  in  Eaves  [l4]. 
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1.3.  Notation  and  Preliminaries 


The  notation  is  developed  here  and  is  used  consistently  throughout 
the  thesis.  To  begin  with  let 

= m-dimensional  euclidean  space  . 

The  following  variables  will  always  represent  vectors  in  euclidean  space; 
a,  b,  u,  v,  w,  X,  y,  z,  and  the  variables  ra,  n and  s will  denote 
their  various  dimensions.  Any  point  x € r'"  should  always  be  thought  of 
as  a column  vector.  The  row  vector  corresponding  to  x will  be  denoted 
by  x^.  If  X G R™  and  y £ r’^  then  let  (x,y)  be  the  (m+n)  column 
vector  whose  first  m components  are  those  of  x and  whose  last  n 
components  are  those  of  y.  The  1th  component  of  a column  vector  x 
will  be  denoted  by  a subscript,  e.g.  x^,  where  as  superscripts  refer  to 
elements  in  a collection  of  vectors.  Thus  n different  vectors  in 
might  be  represented  by  x^,  ...  , x”.  Whenever  possible  the  superscript 
k will  be  used  for  infinite  collections  and  sequences. 

Let  X,  y £ R*^  and  a £ R^  with  a > 0 then 

(x,y,>  = ^i^i  “ inner  product, 

X < y means  x^  < y^  for  each  i = l,..,,n, 

X < y means  < y^  for  each  i = 1, . . , , n, 

Hxil  = [(x,x)  'i^^"’  = euclidean  norm, 

B(x,a)  = open  ball  of  radius  a and  center  x 
= {z  £ r"/1|x-zI1  < al  . 
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Some  special  notation  for  matric'^vS  is  also  needed.  All  matrices 
will  contain  real  numbers.  The  variables  A,  U,  V and  W will  always 
stand  for  matrices  and  I is  reserved  for  the  square  identity  matrix. 

Its  dimension  will  be  implied  by  the  context  (as  will  the  size  of  the 
vector  0).  If  U is  an  (m  x n)  matrix  and  V an  (m  x i)  matrix  then 
by  [U,V]  is  meant  then  (m  x (n  + i) ) matrix  whose  first  n coltimns 
are  those  of  U and  whose  last  i columns  are  those  of  V.  In  this 
context  an  n-vector  may  be  thought  of  as  an  (n  x 1)  matrix.  The  only 
case  in  which  confusion  can  possibly  arise  is  when  U and  V are  real 
numbers  with  U < V.  In  thiii  case  [U,V]  can  also  be  the  closed  interval, 
i.e.  (x  e R^/U  £ X < V),  depending  on  the  context.  The  ith  column  of 
U will  be  denoted  by  U ^ and  the  ith  row  of  U by  . Also  it 

will  sometimes  be  necessary  to  associate  a coliimn  of  a matrix,  say  A, 
with  a vector  x.  This  will  be  done  by  the  notation  A /■  . . Note 
also  that  if  y G then  Uy  is  well  defined  because  y is  a column 
vector.  Also  if  W is  a (n  X n)  matrix  its  determinant  will  be  denoted 
by  det(W).  Finally  define 

||U||  = sup{|ilfy||/y  e R*^  and  ||y||  = 1]  . 

A great  many  different  functions  are  required  for  (a)  the 
property  of  decomposability  in  Chapter  2 and  (b)  the  various  constrained 
optimization  problems  of  Chapter  For  this  reason,  f,  g,  h,  p,  q,  r,  t, 
F,  G,  L,  P and  Q will  always  represent  functions.  Observe  that  if 
h:R"'  ->  r”,  then  for  each  x £ R*”^,  h(x)  generates  a column  vector 
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h(x)  £ whose  ito  components  is  h^(x).  Furthermore,  f,  g,  h,  r 
and  F will  be  related  to  decomnos ability  whereas  p,  q,  r,  t,  G,  P 
and  Q will  be  used  for  the  constrained  optimization  problems.  Point 
to  set  maps  will  be  represented  by  H,  S and  T. 

The  handling  of  seqiieuces  also  requires  some  discussion,  for 

If 

example,  a sequence  of  vectors  will  be  denoted  by  {x“]  where  it  is 
understood  that  K = 0,1, . . . or  k = 1, 2, . . . as  implied  by  the  context. 
A subsequence  will  be  thougnt  of  as  a subset  K of  the  positive  integers 
and  will  be  written  "{x  ],  k £ K" . If  the  given  sequence  converges  to  x 

it  will  be  written  "{x  } — x for  k ->  «"  and  for  a subsequence, 

Ic  It 

"Cx  ; X for  k £ K" . x is  said  to  be  a cluster  point  of  {x  ] iff 

there  is  a subsequence  K such  that  {x  ] x for  k £ K, 

For  sets,  the  variables  X,  Y,  Z and  0 will  always  be  used. 

m 

If  X is  a subset  of  R then 

bd(X)  = boundary  of  X , 
ci(X)  = closure  of  X , 
hull(X)  = conve.x  hull  of  X , 
int(X)  = interior  of  X , 

X*  = the  set  of  all  non-empty  subsets  of  X. 

If  X and  Y are  subsets  of  r”'  and  if  w £ r"'  and  f :X  -4  Y, 

T;X  ->  Y*  then 
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X + Y - [x  + y/x  e X,  y e Y)  , 

X X Y = [ (x,y)/x  t X,  y e Y)  , 

w + X - (w]  + X , 

X \y  - (x  u x/x  Y}  , 

f(X)  ^ ff(x)  ''x  € X]  , 

T(X)  U T(x)  . 

xex 

Theorems,  lemmas,  etc,  are  numbered  sequentially  vdthin  a chapter 
and  the  following  conventions  have  been  adopted: 

D.i. j = jth  definition  of  Chapter  i , 

L.i.J  = lemma  of  Chapter  i , 

T.i.j  = theorem  of  Chapter  i , 

P.i.j  = problem  of  Chapter  i , 

C.i.j  = corollary  of  Chapter  i , 

A.i. j = jth  assumption  of  Chapter  i. 

Some  preliminary  definitions  and  results  will  be  drawn  upon  in 
later  chapters.  These  notions  are  presented  here. 

Definition  1,1.  Let  0 be  an  open  subset  of  R™.  The  function 
h:0  ->  is  said  to  be  differentiable  at  a point  x € 0 iff  there  is 
an  (n  X m)  matrix  Dh(x)  (called  the  Jacobian  matrix)  such  that 

lim  ||h(z)  - h(x)  - Dh(x)(z-x)||  = 0 
llz-xll  -*0 

z £ 0 
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h is  said  to  be  differentiable  on  0 iff  h is  differentiable  at 

each  point  of  0.  When  n is  a fiaiiction  from  n into  R"^,  the 

(l  X n)  matrix  Dh(x)  is  called  the  gradient  of  h at  x and  its 

transpose  is  '.irritten  V'a(x) . The  function  h is  said  to  be  twice 

differentiable  iff  h is  differentiable  and  if  the  mapping  Dh  is  also 

differentiable.  The  second  derivative  of  h at  x is  written  D h(x). 

Suppose  a function  x r'^  -»  R'^  is  differentiable  at  a point 

(a,b)  e r”"'  X R^.  The  (i  x (m  -r  n) ) matrix  lX,i(a,b)  will  frequently 

be  written  [D  Q(a,b',  D Q(a,b)J  vrhere  D Q(a,b'’  is  the  (£  x m) 

X y X 

matrix  corresponding  to  the  derivative  of  Q with  respect  to  the 
variables  in  R™  and  D^Q(a,b't  for  the  same  in  R^,  A similar  idea 
is  applied  to  gradients. 

It  will  often  be  necessary  to  use  the  derivatives  to  obtain 
bounds.  Ortega  and  Rhineboldt  [4l]  provide  several  such  theorems  and 
they  are  stated  for  use  in  this  thesis  as: 

Theorem  1.1.  Let  f:X  ->  R be  differentiable  on  the  convex  open  set 
X c r'”.  Then 

l|f(z)  - f(x)j|  < sup(ilDf(x  + A(z-x^)!|A  e !0,1]]  !!z-x|| 
for  all  z.  X £ X. 

Proof.  See  Theorem  5.2.3  of  Ortega  and  Rhineboldt  [i^l].  || 
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Theorem  1.2.  Let  f;X  ->  be  twice  continuously  differentiable  on 
the  convex  open  set  X 3 R*”.  Then 

||f(z)  - f(x)  - Df(x)(z-x)||  < sup{!!D^f(x  + A(z-x) ) |1/a  [0,1])  lly-x||^  . 

Proof.  See  Theorem  5.3.6  of  Ortega  and  Rhlneboldt  [^1].  (| 

If  the  functions  involved  are  convex  then  a slightly  weedier  notion 
of  differentiation  exists,  (it  is  assumed  that  the  reader  is  familiar 
with  terms  such  as  "convex  function,"  "convex  set,"  etc.  Rockefeller 
[43],  Stoer  and  Witzgall  [30]  and  Mangasarian  [37]  deal  with  most 
of  these  notions.) 

Definition  1.2.  Suppose  0 is  a subset  of  R™  and  h;0  -»  R^  is  convex. 
The  (n  x m)  matrix  W is  a subgradient  of  h at  x £ 0 iff 

h(z)  > h(x)  + W(z-x)  for  all  z £ 0 . 

The  set  of  all  subgradients  of  h at  x is  denoted  by  3h(x). 

There  is  a relation  between  D. l.L  and  D. 1.2  which  is  stated  in 

Theorem  1.3.  Suppose  0 is  an  open  subset  of  r"*.  A convex  function 
h:0  -»  R^  is  differentiable  at  x € 0 iff  dh(x)  = {Dh(x)]. 

Proof.  See  Theorem  25.1  of  Rockafeller  [43].  II 
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Corollary  1.1.  If  a convex  function  h;R°^  -♦  is  differentiable  then 
h(z)  > h(x)  + (Vh(x),  z-x)  for  all  z £ r""  . 

Proof.  By  T.l.j5,  Vh(x)  £ Sh(x)  and  the  result  follows  from  D.1.2.  || 

Some  other  concepts  related  to  convexity  are; 

Definition  1,5.  A convex  function  p:R^  -> U (+  »}  is  proper  iff 
there  is  an  x £ r'^  such  that  p(x)  > - ». 

Definition  l.^t.  The  epigraph  of  a convex  function  p;R”  -»  R^  U {+  oo] 

D1  3- 

is  {(x, z)  £ R X R /z  >p(x)]  and  is  written  epi(p). 

Definition  1.5.  A convex  function  p:R“  ->  R^  U {+  »)  is  closed  iff 
epi(p)  is  a closed  set. 

Definition  1.6.  The  domain  of  finiteness  of  a convex  function 
PtR^^  R^  U {+  oo]  is  {x  £ r"^/p(x)  < oo  ) and  is  written  dom(p). 

Definition  1.7.  The  vector  d £ R*^  is  said  to  be  a common  direction 
of  recession  of  the  convex  function  hrR*''  -»  R*^  iff  there  is  a scalar 
b and  a vector  x £ R*”  such  that 

h^(x  + Xd)  < b for  all  X > 0 and  i = 1, . . . ,n  . 
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Point  to  set  maps  are  required  for  transforming  some  optimization 
problems  into  fixed  point  problems.  Only  certain  classes  of  point  to  set 
maps  can  be  used  in  fixed  point  computation.  Some  of  these  concepts  are 
^ now  developed. 

Let  S:R  -»  (R  ) be  a point  to  set  map. 

Definition  1.8.  S is  said  to  be  convex  iff  S(x)  is  convex  for 
each  X € r'’'. 

Definition  1.9. 
vdienever 

(1)  {x^}  -*  X 

(2)  e S(x^ 

(5)  {y^)  -» y 

then  y € s(x) . 

Definition  1.10.  A point  to  set  map  S is  usable  iff  S is  non-empty 
convex  and  u.s.c. 

Several  results  concerning  point  to  set  maps  were  proved  by  Merrill. 

They  are  stated  as 

Theorem  l.U.  Let  0 be  an  open  subset  of  r"^  and  suppose  t;0  R^  U {+ 
Then  the  point  to  set  map  T:dom(t)  ^ r"^  defined  by  T(x)  = x - btix) 
is  usable. 

Proof . See  Theorem  10.4  of  Merrill  [38].  1| 


S is  said  to  be  upper  semi-continuous  (u.s.c.)  iff 


for  each  k = 1,2, .. . 


00 
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Theorem  1.3.  Let  X be  a subset  of  R°^.  Also  let  3:cl(X)  -♦  (R°^)* 
ajid  TtR**'  \int(X)  -+  (R°^)  be  usable  point  to  set  maps.  Then  the 
point  to  set  map  -♦  (R™)^  defined  by 


S(x' 

if 

X € int(X) 

hull(S(x)  UT(x)) 

if 

X e bd(X) 

T(x) 

if 

X i cl(X) 

is  also  usable. 

Proof.  See  Theorem  2.6  of  Merrill  [38].  || 


A final  definition  which  is  needed  is  that  of  isotonicity. 

Definition  1.8.  A function  htP''^  -»  R^  is  isotone  iff  h(x)  < h(z) 
whenever  x,  z G and  x < z. 
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CHAPTER  2 


DECOMPOSABILITY  IN  FIXED  POINT  PROBLEMS 


2.1.  General  Theory  and  Examples  for  Functions. 

g 

Given  a function  F mapping  a nonempty  subset  Z of  R 

into  itself,  the  fixed  point  problem  is  that  of  finding  a z £ Z 

with  F(2)  = z.  The  basic  idea  behind  decomposability  is  to  place  a 
structure  on  F such  that  a fixed  point  may  be  computed  by  working  in 
a lower  dimensional  space.  Several  examples  of  where  this  structure 
appears  in  applications  is  also  presented.  This  structure  is  described 

by 

Definition  2.1.  Let  Z be  a nonempty  subset  of  R^  and  let  F:Z  -»  Z. 
The  pair  (F,Z)  is  said  to  be  decomposable  iff  there  are  positive 
integers  m and  n whose  sum  is  s,  nonempty  subsets  X of  R°^  and 

Y of  r"^  whose  cross  product  is  Z together  with  functions  f:Z  X, 

g;Z  -^Y  and  h;X  -»  Y such  that  for  each  x £ X, 

(1^  F(x,h(x))  = (f(x,h(x) ),g(x,h(x))) . 

(2)  If  X = f(x,h(x))  then  h(x)  = g(x,h(x))  . 

The  first  condition  states  that  F may  be  decomposed  into  two  separate 
functions  f and  g with  f providing  the  first  m coordinates  of 
F and  g providing  the  remaining  n coordinates.  The  second  condition 
is  a special  relation  between  the  functions  f,  g and  h which  will 


be  used  to  establish  the  connection  between  fixed  points  of  the  lower 
dimensional  problem  and  fixed  points  of  F.  Tlie  lower  dimensional 
problem  will  be  one  of  finding  a fixed  point  in  r'°.  More  specifically, 
defining  the  function  r:X  -> X by  r(x)  = f(x,h(x))  the  next  theorem 
shows  that  any  fixed  point  of  r yields  a fixed  point  of  F.  This 
ther  is 

Theorem  2.1.  If  (F,Z)  is  decomposable  then  x € X satisfies 
X = f(x,h(x))  iff  fx,h(x))  = F(x,h(x))  where  X and  f are  obtained 
from  D.:^.l. 

Proof.  Suppose  first  that  x = f(x,h(x)).  By  property  (2)  of  D.2,1, 
h(x^  =g(x,h(x)).  Thus  (x,h(x';)  = (f  (x,h(x)  .g(x,h(x) ) ) =F(x,h(x)), 
the  last  equality  being  Justified  by  property  (ll  of  D.2.1.  This  takes 
care  of  the  necessary  part  of  the  theorem.  For  the  sufficiency  part 
suppose  (x,h(x))  = F(x,h(x)).  From  property  (l)  of  D.2.1  it  follows 
immediately  that  x = f(x,h(x'i)  as  desired.  || 

Replacing  f(x,h(x))  by  r(x)  one  may  more  easily  see  what 

T.2.1  is  saying.  It  is  saying  that  if  (F,Z)  is  decomposable  then 

finding  a fixed  point  x of  r yields  a fixed  point  of  F namely 

(x,h(x)).  The  importance  of  this  is  that  finding  a fixed  point  of  r 

m s 

involves  working  in  R instead  of  R . Some  conditions  under  which 
F and  r may  be  expected  to  have  fixed  points  is  developed  in 


15 


Corollary  2.1.  Suppose  {F ,Z)  is  decomposable  and  that  f,  g,  h,  X 
and  Y are  obtained  from  D.2.1.  In  addition  suppose  that  f and  h 
are  continuous  and  that  X is  compact  and  convex.  Under  these  conditions 
F has  a fixed  point. 

Proof.  Lit  r;X  X by  r(x)  = f(x,h(x))  for  each  x £ X.  This  function 
is  continuous  since  f and  h are  continuous  and  since  the  composition 
of  continuous  functions  is  continuous.  Now  apply  the  Brouwer  fixed  point 
theorem  to  r to  obtain  the  existence  of  an  x t X such  that  r(x)  = x. 

By  T.2.I,  (x,h(x))  is  a fixed  point  of  F.  || 

The  following  examples  show  the  value  of  this  rather  straight- 
forward observation.  The  first  example,  although  hypothetical,  shows  the 
potential  of  decomposability  by  reducing  an  (n+1) -dimensional  fixed  point 
problem  to  a 1-dimensional  problem.  The  second  example  is  equality  con- 
strained optimization.  The  third  example  is  a partially  linear  systems  of 
equations  and  the  last  is  the  Bilinear  Complementarity  Problem  proposed 
in  Wilson  [5^]- 

Example  2.1.  Let  a,  b £ with  a < b.  Set  X = [a,b]  and 
y = Thus  Z = X X Y = [a,b]  x R^.  F will  be  constructed  to  satisfy 

the  hypotheses  of  C.2.1  in  such  a way  that  the  function  r will  be  a 
mapping  from  [a,bl  into  [a.b'.  Thus  finding  a fixed  point  of  F 
will,  be  reduced  to  finding  a fixed  point  of  r and  that  will  be  a 
1-dimensional  problem.  To  actually  construct  this  F let  f:Z  -» X 
and  h:X  Y be  arbitrary  but  continuous  functions.  Define  g:Z  Y 
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g(z)  = h(f(z)^  for  all  zS  Z.  Finally  let  F(z)  = (f(z),g(z)) 
for  all  z e Z. 

Proposition  2. 1.  (F,Z)  is  decomposable. 

Proof.  Condition  (l'  of  D.2.1  is  true  by  construction  of  F so  only 
condition  (2)  needs  to  be  verified.  To  this  end  let  x € X with 
X = f(x,h(x)).  Applying  h to  both  sides  of  this  yields  h(x)  = h(f(x,h(x))) 
= g(x,h(x')  as  desired.  1! 

Proposition  2.2.  F has  a fixed  point. 

Proof.  (F,Z)  is  decomposable  and  satisfies  the  hypotheses  of  C.2.1, 
thus  F has  a fixed  point.  ll 

From  the  proof  of  C.2.1  it  is  apparent  that  in  order  to  compute  a 
fixed  point  of  F one  need  only  be  computed  for  r:X  -♦  X,  and  finding 
a fixed  point  of  r is  one  of  searching  [a, b]  as  opposed  to  searching 
[a,b]  X 

Notice  that  f and  h were  completely  arbitrary  except  for 
their  continuity,  thus  they  may  be  made  nonseparable,  nondiff erentiable, 
etc.  Also  this  example  shows  that  there  are  problems  (F,Z)  whose 
fixed  points  would  not  normally  be  computable  because  of  the  large 
dimensionality  yet  if  (F,Z)  is  decomposable  in  the  proper  way  one 
can  find  the  fixed  point  in  a l-dimens tonal  apace. 
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Example  2.2.  In  this  example,  a nonlinear  programming  problem  (NLP) 
of  the  form 


(P.2.1)  min  P(z) 

s.t.  G(z)  = 0 
z € R® 

where  P:  R®  -»  R^  and  G:R®  -»  R*^  is  put  into  the  framework  of  decompos- 
ability.  Since  this  was  the  problem  which  motivated  the  concept  of 
decomposability  all  of  Chapter  U has  been  devoted  to  a complete 
theoretical  analysis  of  this  problem.  In  this  section  a connection 
between  the  NLP  and  decomposability  is  established  in  loose  terms  and 
a rigorous  approach  is  presented  in  Chapter  h. 

In  order  to  show  that  P.2.1  is  in  fact  a special  case  of  decompos- 
ability one  must  (a)  find  a fixed  point  problem  which  is  related  to 
P.2.1  and  (b)  show  that  this  fixed  point  problem  is  decomposable. 

Eaves  [11]  and  Merrill  [38]  have  discussed  extensively  the 
fixed  point  formulation  when  the  NLP  is  in  the  form  of  either  (i)  uncon- 
strained optimization  or  (ii)  inequality  constraints  with  the 
existence  of  a point  at  which  all  constraints  are  strictly  feasible. 
Clearly  P.2.1  does  not  fall  into  either  of  these  categories  (even  when 
the  constraints  G(z)  = 0 are  replaced  by  G(z)  < 0 and  -G(z)  < 0). 

So  the  question  becomes  how  to  transform  an  equality  constrained 
problem  into  an  optimization  problem  of  type  (i)  or  (ii)  as  described 
above.  Three  possible  methods  for  doing  this  are  now  presented,  the 
last  of  which  led  to  the  concept  of  decomposability. 
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Method  1.  Constraint  relaxation. 


In  this  approach  the  equality  constrained  problem  Is  replaced 
by  a sequence  of  problems  of  type  (ii).  This  is  done  by  introducing  a 
tolerance  for  the  constraints.  Formally  let  {a  ] be  a sequence  of 
vectors  in  with  a^  > 0 for  each  k = 1,2,...  and  with  (a^)  -» 0 

for  k ->  00.  Consider  the  family  of  optimization  problems. 

min  P(z) 

s.t.  -a^  < G(z)  < a^  for  k = 1,2,... 

z £ R® 

Under  certain  circumstances  each  of  these  problems  is  of  type  (li)  and 
one  could  then  use  existing  methods  to  partially  solve  the  kth  problem. 
Under  additional  circumstances  one  might  expect  the  limit  of  the  solutions 
(assuming  such  a limit  exists)  to  be  a solution  to  P.2.1.  Computational 
results  from  this  approach  reported  by  Merrill  [3^]  were  extremely  dis- 
couraging and  so  this  approach  was  discarded. 

Method  2.  Lagrangian  approach. 

In  this  method  one  defines  the  Lagrangian  function  L:R®  x R*^  R^ 
by  L(z,u)  = P(z)  + (u,G(z)).  One  can  then  show  (see  Mangasarian  (57] 
for  example)  that  a necessary  condition  for  (z,u)  to  solve  P.2.1  is 
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V^L(z,u)  = VP(z)  + u'^DG(z)  = 0 
V^L(z,u)  = G(z)  = 0 


assuming  of  course  that  P and  G are  differentiable  functions  on 
s 

R X R . This  is  a system  of  (n+s)  equations  in  (n+s)  unknowns  v*ich 

may  be  transformed  into  a fixed  point  problem  by  defining  F:R^  x ->  R^xR*^ 

by  F(z,u)  = (z,u)  - (V  L(z,u)  ,VL(z,u)).  This  fixed  point  formulation 

z u 

has  increased  the  dimension  of  the  original  problem  from  s to  (s+n). 

This  is  exactly  contrary  to  the  concept  of  decomposability  and  so  this 
approach  was  also  discarded  for  this  work. 


Method  3.  Decomposability. 

This  approach  is  motivated  by  considering  the  special  form  of 
P.2.1  in  which  the  equality  constraints  are  linear,  for  then  these 
equations  can  be  used  to  solve  for  some  of  the  variables  (called  the  basic 
variables)  in  terms  of  the  remaining  variables  (called  the  nonbasic 
variables).  If  the  rank  of  the  linear  transformation  is  n then  there 
will  be  exactly  n basic  variables  (referred  to  herein  after  by  the 
vector  y = (y^, . . . ^y^^) ) . Correspondingly  there  will  be  r = s-n  non- 
basic variables  (referred  to  by  the  vector  x = (x^, . . . ,x^) ) . These 
observations  are  formalized  in 
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Proposition  2. j.  Let  W Le  an  (n  x s}  matrix  and  w an  n- vector  such 
that  G(z)  - Wz  + w for  each  z £ K^.  If  rankCW)  = n then  there  is  a 
function  h;?."'  -»  such  that  G(x,h(x))  - 0 for  each  x £ R®. 

Proof.  Since  rankCV/)  = n there  are  (n  x m)  and  (n  x n)  matrices  U and 
V respectively  such  that  after  permuting  the  columns  of  W, 

(1)  W --  ru,V]  and  (2)  V is  nonsingular.  Define  h:R”  R*^  by 
h(x)  = + Ux) . To  verify  that  G(x,h(x^)  = 0 note  that 

G(x,hfx))  = V7(x,h(x))  + w 

= [U.V]  (x,h(x) ) + w 
= Ux  + Vh(x)  + w 
= Ux  - (v7  + Ux)  + w 

- 0 . II 

In  the  case  that  the  constraints  of  P.2.1  are  nonlinear,  con- 
ditions may  be  placed  on  the  function  G which  ensures  the  conclusion  of 
Proposition  2.3;  namely,  that  there  is  an  hrR™  ->  R*^  such  that 
G(x,h'|x))  = 0 for  each  x £ r"'.  In  this  case  the  function  h is  an 
implicit  function  and  conditions  for  its  existence  are  established  in 
Chapter  L.  From  here  on, the  function  h will  always  be  the  mapping 
described  above.  Furthermore,  this  function  will  be  used  in  showing 
P.2.1  is  a special  case  of  decoraposability. 

Once  the  existence  of  this  function  h has  been  established  one 
may  then  eliminate  the  basic  variables  (along  with  equality  constraints) 
from  P.2.1.  The  result  is  an  unconstrained  optimization  problem  vdiich 
takes  the  form 
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(P.2.;->) 


min  p(x)  = P(x,h(x))  . 

s.t.  X € R*" 

Since  this  is  a minimization  problem  of  type  (i),  there  is  a fixed  point, 
formulation  of  P.2.2,  naruely  that  of  finding  an  x G such  that 
X - Vp(x)  = X provided  p is  differentiable  (see  Merrill  [38]). 

These  remarks  are  now  summarized  and  stated  in 

Proposition  2,'^.  Let  Z = R®  - r”^  X R*^  and  suppose  there  is  an 
h;R‘“  -»r"  such  that  G(x,h(x))  = 0 for  each  x € R®.  Also  suppose  P, 

G and  h are  differentiable  on  their  respective  domains.  Let  F:Z  -> Z 
be  defined  by  F(x,y)  = (x,y)  - (V^P(x,y)  + 7yP(x,y)^  Dh(x),  G(x,y)). 

Then  (F,Z)  is  decomposable. 

Proof.  Let  X = r“',  Y = R^^  and  define  f;Z  -» X by  f(x,y) 

= x - V^P(x,y)  - c,y)  Dh(x)  and  define  g;Z  Y by  g(x,y)  = y i-G(x.y 
for  each  (x,y)  € Z.  Now  one  may  verify  the  conditions  of  D.n.i  vith 
X,  Y,  f,  g and  h.  || 

Lx  ample Pa:-tially  linear  systems  of  equatio.ns.  In  this  example, 

decoraposability  will  oe  apr.'liec  to  solving  on  s by  s system  of 
equations  in  which  some  of  them  are  linear.  Let  W be  an  (n  x s)  matrix 
and  V be  an  n-vector.  Also  let  EiR”  x R*^  — » R***  be  an  arbitrary 
■^unction.  The  original  problem  may  be  stated  as  that  of  finding  a 

g 

z c.  R such  that 

E(z)  = 0 , 

Wz  + w = 0 , 


The  next  proposition  gives  a condition  under  which  there  is 
a decomposable  fixed  point  problem  vdioss  solution  solves  the  original 
system  of  equations. 

Proposition  2.S.  Let  Z = R®.  If  rank(W)  = n then  there  is  a function 
FiZ  Z such  that  (F,Z)  is  decomposable. 


Proof.  In  order  to  prove  this^  the  appropriate  sets  and  functions  will 
be  created.  In  particular  let  X = r"‘  and  Y = R^.  Since  rank(W)  = n 
there  are  (n  x m)  and  (n  x n)  matrices  U and  V respectively  such 
that  after  rearranging  the  columns  of  W,  (i)  W = [U,V],  and 
(ii)  V is  nonsingular.  Define  f;Z  -♦  X,  g:Z  Y and  h:X  -» Y by 
f(x,y)  = E(x,y)  + x,  g(x,y)  = Ux  + (V  + I)y  + w,  h(x)  = -V“^(Ux  + w) 
for  each  (x,y)  £ Z.  Now  D.2.1  may  be  verified.  I! 

Example  2.h.  The  Bilinear  Complementarity  Problem.  This  problem  arises 

in  economics  and  was  introduced  in  Wilson  [5^]  and  may  be  stated  as  that 

of  finding  x,  y £ r"^  such  that  (a)  x,  y > 0,  (b)  x = Uy  + u eind 

(c)  x.y.  = {V  . y)  for  each  i = 1,  ...,n  where  U and  V are 
1 i .1, 

(n  X n)  matrices  and  u is  an  n-vector.  Through  the  rest  of  this  example 
it  is  assuined  that 

(i)  V is  nonsingular. 

(ii)  = { z £ R*' > 0,  (V  ^.z)  > 0)  is  nonempty  for  each  i = 1, ..., 
(ill)  There  is  no  z £ r'^  with  z > 0 and  v'^z  > 0. 


Let  z'  £ R*'  be  such  that  v'^z'  :■  0,  Now  define  f;R”  x r"  -♦  R°  by 


h(x)  ={ 


^ - 

(V 

^,y);  X r“  ^ R°  by  g(x,y) 

1^1’  • • • ’ 

^^n 

) 

and  h : R^  ->  R^  by 

z € Z^ 

if 

i 

is  the  first  integer  such  that 

z* 

if 

X 

> 0 and  {y  > 0/x  = Uy  + u]  is 

y 

if 

X, 

y > 0,  X = Uy  + u . 

As  usual  define  F;r”  X R*^  ->  r"  x R^  by 


F’''x,y)  = (f(x,y),  g(x.y))  for  all  (x,y)  € r"^  x r"^ 


Proposition  2.6.  If  Z = R^  then  (F,Z)  is  decomposable. 

Proof . Let  X = Y - R . Then  these  sets  together  with  the  functions 
f,  g and  h will  be  shc.m  to  satisfy  D.2.1.  Only  condition  (2)  of 
D.2.1  neeus  verification  so  let  x t r”  with  f(x,h(x))  = x.  Then  by 
construction, 

(x^h^(x)  + h(x)) \ - (V_^,h(x)))  = (x^,. 

and  hence 


x^h^(x^  = ^V_^,h(x'),  , x^h^(x)  = (V  ^,h(x)> 


(xihi(x),  ...  , x^h^(x))  = v\(x) 


2h 


so 


T -1 

Multiplying  both  sides  of  this  by  (V  ) yields 

h(x)  ^ (V^)“^  (x^h^(x),  ...  , x^h^(x))  = g(x,h(x))  . |1 

Proposition  2.7.  If  f(x,h(x))  = x then  x > 0. 

Proof.  Suppose  not.  Then  there  is  a J between  1 and  n such  that 
X.  < 0.  Let  i be  the  first  subscript  with  x.  < 0.  By  the  definition 
of  h,  h(x)  = z where  (V  ^,7.)  >0  and  > 0.  From  the  fact  that 
f(x,h(x))  = X it  follows  that  x^h^(x)  = (V  ^,h(x))  but  the  left 
side  is  strictly  negative  and  the  right  side  is  greater  than  or  equal  to 
0.  This  contradiction  shows  the  claim.  |i 

Proposition  2.8.  If  f(x,h(x))  = x then  h(x)  > 0. 

Proof.  Suppose  not.  Then  there  is  an  i between  1 and  n such  that 
h^(x)  <■  0.  From  Proposition  2.7  it  may  be  assumed  that  x > 0 and  con- 
sequently  that  h(x)  = z'  where  V z’  > 0.  By  the  fact  that  f(x,h(x))  = 
it  follows  that  x^h^(x)  = (V  j^,h(x))  but  the  right  side  is  strictly 
positive  and  the  left  side  is  less  than  or  equal  to  0.  This  contradiction 
chows  the  claim.  || 

Proposition  2.9.  If  (x,h(x) ) =F(x,h(x))  then  (x,h(x))  solves  the 


BLCP. 


Proof.  From  Propositions:  .('and  .3  both  x and  h(x)  are  nonnegative. 


Using  the  fact  that  f(x,h(x))  = x one  may  conclude  that  x^h^(x)  = 
,h(x))  for  each  i = 1,  ...,n.  Finally  since  h(x)  >0  and  by 
assumption  (iii)  it  follows  that  x = Uh(x)  + u.  Thus  in  fact  (x,h(x)) 
solves  the  BLCP.  i| 

In  this  section  the  concept  of  a decomposable  fixed  point  problem 
was  discussed  and  several  examples  of  this  property  were  presented.  The 
most  interesting  example  was  optimization  with  equality  constraints.  One 
would  like  to  be  able  to  handle  both  equality  and  inequality  constrained 
problems,  however,  in  order  to  do  this  it  appears  necessary  to  enter  the 
framework  of  point  to  set  maps.  The  next  section  is  devoted  to  this 
extension. 


2 . P . General  Theory  and  Examples  for  Point  to  Set  Maps. 

g 

Given  a nonempty  subset  Z of  R and  a point  to  set  map 
S:Z  ->  Z*  the  fixed  point  problem  is  that  of  finding  a z € Z such  that 
z S(z).  The  basic  idea  is  to  put  some  structure  on  S which  allows 
one  to  solve  an  equivalent  problem  but  in  a lower  dimensional  setting. 

This  property  is  a straightforward  generalization  of  the  one  described 
in  the  previous  section.  Corresponding  to  D.2.1  is 

Definition  2.2.  Let  Z be  a nonempty  subset  of  and  let  S:Z  ->  Z* 

be  a nonempty  point  to  set  map.  The  pair  (S,Z)  is  said  to  be  decomposable 
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Iff  there  are  positive  integers  m and  n whose  sum  is  s,  nonempty- 

subsets  X of  r”*  and  Y of  whose  cross  product  is  Z together 

with  nonempty  point  to  set  maps  S„:Z  ->  X*-,  S :Z  -»  Y*  and  a function 

r g 

h:X  -7  Y such  that  for  each  x u X, 

(1)  G(x,h(x^)  = S^(y.,h(x))  v Sg(x,h(x)). 

(P)  If  X £ 3„‘!x,h(x))  then  h(x)  £ S (x,h(x))  . 

^ s 

The  first  condition  states  that  at  each  x £ X the  set  S(x,h(x))  may 

be  expressed  as  the  cross  product  of  the  sets  S (x,h(x))  and  S (x.h(x)) 

^ 6 

The  second  condition  is  a special  relation  between  S„,  S and  h ’^diich 

f g 

will  be  used  to  establish  the  connection  betv/een  fixed  points  of  the 
lower  dimensional  problem  and  fixed  points  of  S. 

One  would  expect  that  D.2.2  reduces  to  D.2.1  in  the  special  case 

^Aiere  S(z)  is  a set  consisting  of  a single  point  for  each  z £ Z. 

This  is  established  in 

Proposition  2.10.  Let  F;Z  -»  Z and  define  the  point  to  set  map 
S:Z  -)  Z*  by  S(z)  = {F(z')}  for  each  z £ Z.  Then  (F,Z)  is  a 
decomposable  function  iff  (F,Z)  is  a decomposable  point  to  set  map. 

Proof.  Assume  first  that  (F,Z)  is  a decomposable  function  and  let 
X,  Y,  f,  g and  h be  obtained  from  D.2.1.  Define  the  point  to  set 
maps  S^:Z-*X’<-  and  by  S^(z)  =[f(z))  and  Sg(z)  ={g(z)) 

for  each  z £ Z.  It  is  now  an  easy  matter  to  verify  that  X,  Y,  S_,  S 

^ O 
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and  h satisfy  D.2.L?.  This  takes  care  of  the  necessary  part  of  the 
proposition.  To  go  the  other  way  suppose  (S,Z)  is  a decomposable 
point  to  set  map.  Let  X,  Y,  S„,  S and  h be  obtained  from  D.2.2. 

i G 

To  shftw  that  (F,Z)  is  decomposable,  functions  f and  g will  be  con- 
structed in  such  a way  that  together  with  X,  Y and  h they  will  satisfy 
D.2.1.  Simply  define  f(z)  to  be  any  element  of  S^(z)  and  similarly 
for  g(z).  To  verify  condition  (l)  of  D.2.1  let  x £ X.  Then  from  the 
decomposability  of  S, 

F(x,h(x))  £ {F(x,h(x)))  = S(x,h(x))  =■  S_(x,h(x))  x S (x,h(x)) 

= {(f(x,h(x)),  g(x,h(x1))]  , 

the  last  equality  being  Justified  because  S-(x,h(x))  and  S (x,h(x)) 

^ 8 

are  sets  with  only  one  point.  Hence  F(x,h(x))  = (f(x,h(x)),  g(x,h(x))). 
Condition  (2)  is  trivial  to  verify.  || 

This  result  has  shown  that,  on  the  surface,  D...-;.2  is  the  proper 
generalization  of  D.2.1.  The  next  theorem  is  the  analog  to  T.2.1. 

Theorem  2.2.  If  (S,Z)  is  decomposable  then  x -T  X satisfies 
X £ S^(x,h(x))  iff  (x,h(x))  £ S(x,h(x))  where  X and  are  obtained 

from  D.  1. 2. 

Proof.  Suppose  first  that  x S^(x,h(x)).  By  property  (2)  of  0.2.^;, 
h(x)  £ S (x,h(x)).  Thus  (x,h(x))  £ S„(x,h(x))  x S (x,h(x))  = S(x,h(x)), 
the  last  equality  being  Justified  by  property  (1)  of  D.,.'.2.  This  takes 
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care  of  the  necessary  part  of  the  theorem.  For  the  sufficiency  part, 
suppose  fx,h(x1)  e S(x,h(x)).  From  property  (l)  of  D.2.2  It  follows 
immediately  that  x £ S^(x,h(x))  as  desired,  !| 

Defining  the  point  to  set  map  S^:X  ~>X*  by  S^(x)  = S^Cxjhfx)) 
one  may  more  easily  see  what  T.2,2  is  saying.  It  is  saying  that  if 
(S,Z)  is  decomposable  then  finding  a fixed  point  x of  yields  a 

fixed  point  of  S namely  (x,h(x)).  The  importance  of  this  is  that 
finding  a fixed  point  of  involves  working  in  r"™  instead  of  R^. 

The  next  task  is  to  develop  some  conditions  under  which  S and  S 

r 

may  be  expected  to  have  fixed  points. 

Corollary  2.2.  Suppose  (S,Z)  is  decomposable  and  that  in  addition 
Sj,  is  usable  (i.e.  nonempty  convex  and  upper  semi-continuous).  Suppose 
also  that  h is  continuous.  If  X is  compact  and  convex  with  int(X) 
nonempty  t.hen  S has  a fixed  point. 

Proof.  Define  S^:X  -^X-f-  by  S_(x')  = S^(x,h(x)).  is  a usable 

point  to  .set  map  since  the  composition  of  usable  maps  is  usable  (see 
Theorem  1’,  p.  11:5  of  Berge  '31).  Now  apply  the  Kakutani  fixed  point 
theorem  (2'^]  to  to  obtain  the  existence  of  an  x t X such  that 

X ■ S^Cx).  By  T.2.^,  (x,h(x))  is  a fixed  point  of  S.  || 

Two  examples  of  the  property  of  decomposability  are  discussed. 

The  first  example,  although  hypothetical,  shows  the  power  of  decomposability 
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by  reducing  an  (n+1) -dimensional  fixed  point  problem  to  a 1-dimensional 
problem.  The  second  example  is  optimization  under  both  equality  and 
inequality  constraints. 

Example  2.^.  Let  a,  b t with  a < b.  Set  X = la,b]  and  Y = R^. 
Thus  Z = X X Y = [a,b]  x R*^.  S will  be  constructed  to  satisfy  the 
hypotheses  of  C.2.2  in  such  a '«ra,y  that  the  point  to  set  map  will  be 

a mapping  from  [a,b]  into  fa,b]*.  Thus  finding  a fixed  point  of  S 
will  be  reduced  to  finding  a fixed  point  of  S^,  and  that  will  be  a one- 
dimensional problem.  To  actually  construct  this  S let  h:X  ^ Y be 
an  arbitrary  continuous  function  and  let  S^:Z  X*  be  any  usable  point 

to  set  map.  Define  S ;Z  ->  Y*  by  S (x,y)  = h(S_(x,y)),  Finally  define 

6 el 

S;Z  -*  Z*  by  S(x,y)  = S„(x,y)  X S (x,y)  for  each  (x,y)  £ Z. 

I s 

Proposition  2.11.  (S,Z)  is  decomposable. 

Proof.  Condition  (l)  of  D.2.2  is  true  by  construction  so  only  condition 

(2)  needs  to  be  verified.  To  this  end  let  x £ X with  x £ S^(x,h(x)). 

Applying  h to  both  sides  yields  h(x)  £ h(S„(x,h(x) ) ) = S (x,h(x)) 

1 6 

as  desired.  || 

Proposition  2.12.  S has  a fixed  point. 

Proof.  (S,Z)  is  decomposable  and  satisfies  the  hypotheses  of  C.2.2, 
thus  S has  a fixed  point.  || 
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From  the  proof  of  C.2.2  it  is  apparent  that  in  order  to  compute 
a fixed  point  of  S,  one  need  only  compute  a fixed  point  of  S^:X  -» X* 
and  this  is  a problem  of  searching  la,b]  as  opposed  to  [a,b]  x 
Notice  that  and  h were  completely  arbitrary  except  for  their 

continuity  properties.  This  example  shows  that  there  are  probl-ems  (S,Z) 
whose  fixed  points  would  not  normally  be  computable  because  of  the  high 
dimensionality  yet  if  (S,Z)  is  decomposable  in  the  proper  way,  one  can 
find  the  fixed  point  in  a 1 -dimensional  space. 

Example  2.6.  With  the  concept  of  a decomposable  point  to  set  map  it  is 
possible  to  show  that,  under  certain  circumstances,  the  general  nonlinear 
programming  problem  of  the  form 

min  P(z)  P;R®-»R^ 

s.t.  g(z)  =0  where  G;R®--*r’^ 

Q(z)  <0  Q;R®  ->  R^ 


may  be  set  up  as  a decomposable  point  to  set  map  fixed  point  problem. 

In  order  to  do  this,  many  of  the  concepts  developed  in  Chapter  4 are 
required  and  in  order  to  avoid  duplication,  a proof  of  exactly  how  this 
can  be  done  is  postponed  until  Appendix  A.  Recall  however,  that 
decomposability  is  the  property  of  being  able  to  solve  a particular  fixed 
point  problem  by  solving  a lower  dimensional  fixed  point  problem,  so  in 
order  to  apply  this  concept  to  the  nonlinear  programming  problem  it  will  be 
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necessary  to  (a)  find  a fixed  point  problem  vdiioh  is  related  to  the  NLP 
and  (b)  show  that  this  fixed  point  problem  is  decomposable.  Chapter  1 
deals  with  (a)  and  Appendix  A deals  with  (b).  The  approach  is  very  much 
related  to  the  one  developed  in  Example  2.L’. 

In  this  section  the  concept  of  a decomposable  point  to  set  map 
was  discussed  and  several  examples  were  presented.  The  next  section  is 
a further  generalization  of  these  notions. 


2.3.  Generalizations . 

A generalization  of  decomposability  for  f'onctions  and  then  for 
point  to  set  maps  is  developed. 

Definition  2.3.  Let  Z be  a nonempty  subset  of  and  let  F:Z  Z. 

The  pair  (F,Z)  is  weakly  decomposable  iff  there  are  positive  integers 


m and  n whose  sum  is 

s,  nonempty  subsets  X of 

^m 

and 

Y of 

together  with  functions 

f:Z  -4  X,  gtZ  ^ Y,  h;X  -»  Y 

and 

c:X 

X Y ^ Z 

such  that  for  each  x £ X, 

(1)  F(c(x,h(x)))  = c(f(c(x,h(x))),g(c(x,h(x)))). 

(2)  If  X = f(c(x,h(x)))  then  h(x)  = g(c(x,h(x) ) ) . 

(3)  c is  1-1. 

Note  immediately  that  when  c is  the  identity  map  on  X x Y this 
definition  is  exactly  that  of  decomposability  of  (F,Z).  Whenever 
(F,Z)  is  weakly  decomposable,  X,  Y,  f,  g,  h and  c will  refer  to 
the  corresponding  sets  and  functions  derived  from  D.2.5. 
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As  one  would  expect  there  are  straightforward  generalizations 
of  T.2.1  and  C.2.1  which  may  be  stated  as 

Theorem  2.3.  If  (F,Z)  is  weakly  decomposable  then  x € X satisfies 
X = f(c(x,h(x)))  iff  c(x,h(x))  = F(c(x,h(x) ) ) . 

Proof.  Suppose  first  that  x = f (c(x,h(x) ) ) . By  condition  (2)  of 
D.2.3,  h(x)  = g(c(x,h(x) ) ) . Thus 

c(x,h(x))  = c(f (c(x,h(x) ) ) , g(c(x,h(x) ) ))  = F(c(x,h(x) ) ) , 

the  last  inequality  being  justified  by  property  (1)  of  D.2.3.  This  takes 
care  of  the  necessary  part  of  the  theorem.  For  the  sufficiency  part, 
suppose  c(x,h(x))  = F(c(x,h(x) ) ) . From  property  (2)  of  D.2.3, 
F(c(x,h(x)))  = c(f (c(x,h(x) ) ),  g(c(x,h(x) ) ))  and  since  c is  1-1 
it  follows  that  x = f(c(x,h(x) ) ).  || 

Corollary  2.3.  Suppose  (F,Z)  is  weakly  decomposable.  Suppose  in 
addition  that  f,  h and  c are  continuous  and  that  X is  compact 
and  convex.  Then  F has  a fixed  point. 

Proof.  Define  r:X  -> X by  r(x)  = f(c(x,h(x)))  for  each  x £ X.  Note 
that  r is  continuous  because  f,  c and  h are  and  because  the 
composition  of  continuous  functions  is  continuous.  Now  apply  the  Brouwer 
fixed  point  theorem  to  r to  obtain  the  existence  of  an  x £ X such 
that  r(x)  = x.  By  T.2.5,  c(x,h(x))  is  a fixed  point  of  F.  || 
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A similar  concept  exists  for  the  point  to  set  map  case. 


Definition  2.^.  Let  Z he  a nonempty  subset  of  R®  and  let  S:Z  ->  Z* 
be  a point  to  set  map,  Tlie  pair  (S,Z)  is  weakly  decomposable  iff  there 
are  positive  integers  m and  n whose  sum  is  s,  nonempty  subsets  X 
of  r”  Sind  y of  R*\  functions  h:X  Y and  c;X  x Y -»  Z together 
with  point  to  set  maps  S^:Z  ->  X^  and  S^rZ  -»  Y*'  such  that  for  each 
X t-  X, 

(1)  S(c(x,h(x)))  = c(S^(c(x,h(x) ) X Sg(c(x,h(x) ) ) ) , 

(2)  If  X € S„(c(x,h(x) ) ) then  h(x)  £ S (c(x,h(x))). 

^ 8 

(3)  C is  1-1. 

Note  that  when  c is  the  identity  on  X x Y this  definition  is  exactly 

that  of  decomposability  for  (S,Z).  Whenever  (S,Z)  is  weakly  decomposable, 

X,  Y,  S-,  S , h and  c will  refer  to  the  corresponding  sets,  point  to  set 
^ S 

maps,  and  functions  of  D.'^.h. 

As  one  would  expect  there  eo'e  straightforward  generalizations  of 
Proposition  2.10,  T.2.3  and  C.2.3.  They  are  stated  as 

Proposition  2.13.  Let  F:Z  ->  Z and  define  the  point  to  set  map  S:Z  -♦  Z* 
by  S(z)  = [F(z))  for  each  z £ Z.  Then  (F,Z)  is  a weaJtly  decomposable 
function  iff  (S,Z)  is  a weakly  decomposable  point  to  set  map. 

Proof . Assume  first  that  (F,Z)  is  a weakly  decomposable  function  and 

let  X,  y,  f,  g,  h and  c be  the  sets  and  functions  obtained  from  D.2.3. 

Define  the  point  to  set  maps  S^:Z  -»  X*  and  -►  Y*  by  S^(z)  ={f(z)) 

and  S (z)  = {g(z))  for  each  z £ Z.  It  is  now  an  easy  matter  to  show 
8 

that  X,  Y,  S„,  S , h and  c satisfy  D.2.ii.  This  takes  care  of  the 

o 


necessary  part  of  the  theorem.  To  go  the  other  v/ay  suppose  (S,Z)  is 


a weaJii.y  decomposable  point  to  set  map.  Let  X,  Y,  h,  c,  and 
be  the  sets,  functions  and  point  to  set  maps  obtained  from  D.2.4.  To 
show  tho.t  (F,  Z'  is  a weakly  decomposable  function,  f and  g will 
be  constructed  in  such  a way  that  together  with  h,  c,  X and  Y they 
satisfy  D.2.3.  Simply  define  f(z)  to  be  any  element  of  S^(z)  and 
similarly  for  g(z).  To  verify  condition  (l)  of  D.2.5  let  x Y X. 

Then 

F(c(x,h(x)))  € {F(c(x,h(x) ) ) ) 

■-  S(c(x,h(x))) 

= c(S^(c(x,h(x) ) ) X 3g(c(x,h(x)))) 

= c({f(c(x,h(x''))]  X (g(c(x,h(x'' ))} 

€ {c(f(c(x,h(x))),  g(c(x,h(x) ) ))■! 
so 

F(c(x,h(x)))  (f  (c(x,h(x' ) ),  g(c(x,h(x) ) ) ) 

as  desired.  Condition  (2)  of  D.2.5  is  trivial  to  verify  and  this  completes 
the  proof.  || 

Theorem  2. If  (S,Z)  is  weakly  decomposable  then  x Y X satisfies 
x ' S^(c(x,h(x)))  iff  c(x,h(x))  € s(c(x,h(x) ) ) . 

Proof.  Suppose  first  that  x G S^(c(x,h(x) ) ) . By  property  (P)  of  D.P.U, 

h(x)  G S ^cCx.h^x) ) ) . Thus 
8 

c(x,h(x))  G c(S^.(c(x,h(x)))  x Sg(c(x,h(x'' ) ) ) = S(c(x,h(x)))  , 


the  last  equality  being  Justified  by  condition  (l)  of  D.2,l4.  This  takes 
care  of  the  necessary  part  of  the  theorem.  For  the  sufficiency  part 
suppose  c(x,h(x))  (Z  S(c (x, h(x) ) ) . From  condition  (l)  of  D.2.4  and 
the  fact  that  c is  1-1  it  follows  that  x C S(c(x,h(x)))  as  desired.  || 

Corollary  2.^.  Suppose  (S,Z)  is  weakly  decomposable.  Suppose  in 
addition  that  is  a usable  point  to  set  map  and  that  h and  c 

are  continuous.  Suppose  also  that  X is  compact  and  convex  with 
int(X)  nonempty.  Under  these  conditions  S has  a fixed  point. 

Proof.  Define  S^;X  X*  by  S^(x)  = S^(c(x,h(x) ) ) . is  a usable 

point  to  set  map  since  the  composition  of  usable  maps  is  usable  (see 
Theorem  1',  p.  115  of  Berge  [5]).  Now  apply  the  Kakutanl  fixed  point 
theorem  [28]  to  to  obtain  the  existence  of  an  x L X with  x t S^(x). 

By  T.2.U,  c(x,h(x))  is  a fixed  point  of  S.  11 

This  chapter  has  dealt  with  the  structure  of  decomposability  as 
it  applies  to  functions  and  point  to  set  maps.  Several  examples  of  this 
were  pointed  out,  the  most  interesting  of  which  is  optimization  with 
both  equality  and  inequality  constraints.  The  next  step  is  to  find  a 
method  for  solving  these  problems.  Chapter  5 develops  an  algorithm 
which  may  sometimes  be  used  to  solve  the  fixed  point  problem  and  Chapter 
shows  how  to  convert  the  optimization  problem,  into  a fixed  point  problem. 


CHAPTER  3 


AN  ALCXjRITHK  FOR  COMFJTING  FIXED  POINTS 


3.1.  Triangulations. 

In  this  chapter  an  algorithm  which  attempts  to  compute  fixed 
points  of  certain  point  to  set  maps  is  described.  It  was  developed  by 
Merrill  [38]  in  1971.  A slightly  more  general  version  was  independently 
and  simultaneously  developed  by  Eaves  and  Saigal  [15]-  In  very  general 
terms  the  algorithm;  attempts  to  compute  the  fixed  points  of  a sequence 
of  piecewise  linear  (PL)  functions  approaching  the  original  function. 

Under  certain  hypotheses  each  of  these  points  may  be  computed  in  a 
finite  number  of  steps.  Under  additional  circuiristances  there  will  be 
a cluster  point  of  the  sequence  and  that  will  be  the  desired  fixed 
point.  To  use  this  approach  it  is  necessary  to  have  (a)  a computerized 
method  for  generating  the  sequence  of  PL  approximations,  (b)  a systematic 
procedure  for  attempting  to  move  through  these  pieces  of  linearity  in 
search  of  a fixed  point  and  (c)  some  conditions  under  which  one  might 
expect  the  method  to  find  such  points  in  a finite  number  of  steps. 

These  are  the  three  sections  of  this  chapter. 

Let  M be  a nonempty  subset  of  R*^  and  S:M  •-»  (R'’’)  be  a 
usable  point  to  set  map  whose  fixed  point  is  sought.  In  order  to  construct 
a PL  approximation  f^  to  S it  will  be  necessary  to  break  up  M into 
pieces  (called  simplexes)  which  fit  together  in  a very  special  way  and 
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on  which  will  actually  be  linear  and  continuous.  This  partition 

of  M is  called  a triangulation.  Before  rigorously  defining  this 
concept  it  will  be  helpful  to  understand  its  building  blocks,  namely, 
the  simplexes. 

Definition  3.1.  For  each  i = define  an  1- simplex  in  r’°  to 

be  the  convex  hull  of  (i+1)  points  of  r”  in  general  position.  Given 

these  points,  say  x^,...,  x^,  the  i-simplex  is  then  hull({x^, . . . ,x^) ) . 

Let  T = hull({x^, . . . ) be  an  m-simplex  of  R*”.  The  points 

x^,...,x*’'  are  called  the  vertices  of  x.  Note  that  they  are  actually 

0-simplexes.  One  can  also  generate  very  natural  1-simplexes  from  x 

^1 

by  considering  any  pair  of  vertices  say  x and  x and  then  forming 
il  ig 

hull({x  , X j).  In  general  one  can  generate  for  each  j = 0, . . . , i a 
j- '’implex  from  x by  choosing  any  subset  of  (j+1)  of  the  original 
vertices,  say  x , . . . , x and  forming  hull({x  , ...  , x 'J)). 

These  are  called  the  .j-faces  of  x.  With  these  concepts  in  hand  the 
notion  of  a triangulation  is  quite  understandable. 

Definition  3.2.  Let  M be  a nonempty  subset  of  and  97^  be  a 

finite  or  countable  collection  of  m-simplexes.  Let  ^i  for  i = 0, . . . ,m 
be  the  set  of  i-faces  of  members  of  ^ . Then  is  a triangulatlon 
of  M iff 

(1)  M=Lt{x/xe^]. 

(2)  Each  pair  of  m-simplexes  are  either  disjoint  or  meet  in  a common  face. 
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(.'5)  Each  (e-I)  simplex  in  belongs  to  at  most  two  m-simplexes. 

(Those  \.m-l)  siiaplexes  which  belong  to  exactly  one  ra-sitnplex  are 
called  boiindary  (m-l'' -simplexes. 'i 

(U)  Each  point  in  M has  a neighborhood  meeting  only  finite  many 
m-simplaxes  of  . 

Property  (l)  states  that  M is  the  union  of  all  the  :n-simp1exes 
in  Properties  [2)  and  (p)  describe  how  these  m-simplexes  must  fit 

together  and  property  (b)  guarantees  that  each  bounded  subset  of  M 
meets  cmly  finitely  many  m-simplexes  of  0/}  This  is  needed  for  the 
proof  of  finite  convergence.  A generalization  of  this  notion  is  presented 
in  Saves  [lb’  and  other  pertinent  references  include  Cairns  [tl,  Todd 
[51.‘o2].  Kuhn  Scarf  and  Hansen  [25]. 

The  trian.gulation  /5y  can  be  used  to  generate  a PL  approximation 
to  S through  the  use  of  the  vertices  First  consider  a map 

-»  defined  by  f(x)  t S(x)  for  each  x € ^q.  There  is  a 
unique  extension  of  f to  a PL  map  defined  by 

, m . . 

f^('t;  = Z f(x^) 

i-O 

where  x h t = hull({x^,  ...  , x"']  [ £ and  x = ^--q  Note  that 

by  property  (2)  of  D.5.2,  f^  is  well  defined  since  each  x c M has  a 
unique  representation  in  this  form.  Furthermore,  f^  is  linear  on 
each  T ''  This  map  is  called  a PL  approximation  to  S induced 

by  O/j  . A measure  of  how  closely  approximates  S is  given  by 

raesh(^)  - sup{rnax(  ||u-vl| 'u,  v £ t)  ''t  £ ^). 
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The  algorithm  is  going  to  attempt  to  find  a fixed  point  of 
If  it  is  successful  it  will  terminate  with  an  m-simplex 
T = hull((x*^, . . . jx'”} ) and  an  x £ t with  f^(x)  = x.  This  may  be 
expressed  as  the  existence  of  multipliers  }P , ...  , such  that 


(f(x°)  - x°.l), 


, (f(x"')  -x^l)]  (A°,...,X'")  = (0,...,0,1) 


/'.  > 0 for  i = 0,  1,  ...  , m. 


Setting  X = yields  f^(x)  = x.  Such  an  m-simplex  is  said 

to  be  completely  labelled.  The  next  question  is  how  to  systematically 
move  through  the  triangulation  in  search  of  this  special  simplex.  This 
is  the  topic  of  the  next  section. 


.5.2.  Moving  Through  a Triangulation. 

No  direct  search  procedure  was  found;  instead,  a method  based 
on  a triangulation  of  r'"  x 10,1]  (in  which  all  the  vertices  belong  to 
either  r"'  x {Ol  or  R™  x [!))  was  developed.  An  example  of  such  a 
triangulation  is  depicted  in  Figure  J.l  for  the  case  m = 1. 


FIGURE  3.1 


It  is  not  hard  to  see  that  the  eollaction  of  ra-simplexes  lying  in 
r'*'  X (0)  induces  a triangulation  of  r'"  as  does  those  in  x {li. 

If  T is  such  an  m-simplex  let,  (t)'  denote  the  induced  m-slraplex  in 
r''^  obtained  by  di-opping  the  last  coordinate  of  each  point  of  t. 

Recall,  that  the  objective  is  to  find  a completely  labelled 
simplex  in  r'^I  The  algorithm  will  be  designed  in  such  a way  that  if 
there  is  such  an  ni-simplex  it  will  lie  in  r''''  x (IJ.  More  specifically 
the  algorithm  will  start  with  a special  m-simplex  of  r"'  x {Oj. 

It  will  then  generate  a (possibly  infinite)  sequence  of  "adjacent" 

(m+1)  and  m-simplexes  x^.  ...  in  such  a vray  that  if 

lies  in  x (1)  then  (x*,'  is  the  desired  ra-simplex  (see  Figure 
5.2). 

r”"  X (1) 

r”  X {0} 

FIGURE  3.2 


4 

T 


This  movement  will  now  be  made  mathematically  rigorous. 

Fix  a triangulation  ^ of  R*"  x [0,1]  such  that  each  vertex 

may  be  expressed  as  a vector  (x,u)  where  x € R®  and  u t (0,1]. 

For  each  such  vertex  define  a column  vector  A , . _ r''^^  by 

.vx.u) 


(f(x)-x,  1) 


if  u ^ 1 


A 


(x.u) 


(w-x, 1) 


if  u = 0 


where  (w,0)  t.  R™  X (0]  belongs  to  the  interior  of  a unique  m-simplex 
This  vector  w is  called  the  starting  point,  since  it  is  from 
this  initial  simplex,  i®,  that  the  search  will  begin.  Finding  an  m-simplex 
in  r'”  ^ (1)  which  contains  a linear  approximate  fixed  point  is  equivalent 
to  finding  a basic  solution  to 


{*)  AX  = (0,...,0,1) 


X^  > 0 for  i = 0, . . , , m 


such  that  the  columns  of  A which  form  the  basis  corresponds  to  vertices 
of  an  m-simplex  in  R™  x f 1] . 

To  start  the  search  for  this  m-simplex  compute  a basic  feasible 
solution  to  this  system  of  equations  which  uses  columns  of  A correspond- 
ing to  the  unique  m-simplex  - hull({  (x*^,0) , ...  , (x'",0)])  in 
R*^  X {0]  containing  the  starting  point  (w,0).  The  reader  will  find 
it  useful  to  refer  frequently  to  Figure  3.3. 


(x"’,l)  (x^,l) 


I Is  a boundary  m-simplex  of  the  triangulation  and  by  property 

('■))  of  Do. 2,  it  belongs  to  exactly  one  (mt-l) -simplex,  namely,  cr*^.  Let 

(x"'^^,  1)  be  that  vertex  which  when  , -joined  to  produces  cP . In 

Figure  j.3  this  vertex  is  (x'  ,!)•  Proceed  by  bringing  ^ , 

into  the  basis.  Some  column  will  drop  from  the  old  basis.  This  vertex 

corresponds  to  one  of  the  original  vertices  (x^,0),  ...  , (x'”,0)  say 

(x‘',0).  Let  be  the  m-simplex  obtained  fiom  by  drojjping  (x'^,0). 

In  Figure  3.;;  this  corresponds  to  moving  from  to  after  having 

dropped  (x*^,0).  By  property  (3)  of  a triangulation  since  is  not  a 

boundary  ra-simplex  it  must  belong  to  exactly  two  (m+1^ -siraplexes.  One 
0 1 

of  them  is  o'.  Call  the  other  one  o'  . This  again  corresponds  to  a 


m-+2 


new  vertex  (x  ^,ul  for  which  A 


. (x'n+^’,u) 


may  be  brought  into  the 


basis.  Continuing  in  this  manner  the  sequence  of  m and  (m+1) -simplexes 
0 0 1 

r , 0 , I , . . . IS  generated  and  each  m-siraplex  has  a basic  solution  to 
{*).  Now  only  one  of  four  things  can  happen  -with  this  secuence. 


Case  1.  The  algorithm  repeats  an  (m+1) -simplex  to  which  it  has  already 
been.  Merrill  [3^M  has  shown  that  this  cannot  happen  if  the  dropping 
vertex  is  chosen  by  a lexicographic  resolution  technique  such  as  proposed 
in  Dantzig  [9]  or  Gale  fPO],  Hence  cycling  may  be  avoided  and  this  case 
need  not  be  considered. 


Case  P.  The  algorithm  hits  a boundary  m-simplex  in  R*”  x (O).  This  case 
cannot  arise  becuase  the  only  m-simplex  in  r"*  (0)  with  a solution  to 

(’*'1  is  and  Case  1 rules  out  the  possibility  of  ever  returning  to  (p , 


Case  The  aLgoritVim  hits  a boundary  m-simplex  in  r'"  X ( 1) . In  this 

case  the  algorithm  has  succeeded  and  this  m-simplex  is  the  one  containing 
the  linear  approximate  fixed  point. 

Case  An  infinite  sequence  of  distinct  ra  and  (ra+1' -simplexes  is 
generated.  In  thi.s  case  the  algorithm  is  said  to  have  failed. 

Since  Case  i and  Case  cannot  occur,  one  would  like  some  conditions 
on  the  point  to  set  map  S which  ensures  that  Case  cannot  arise.  Then 
only  Case  "5  can  happen  and  this  is  what  is  desireii.  Before  proceed in^^ 
to  give  some  sufficient  conditions  on  S which  rule  out  Case  t,  a subtle 
point  must  be  cleared  up.  In  describing  the  algorithm  it  was  stated 
that  a new  column  A will  be  brought  into  the  existing  basis. 

The  fact  that  this  may  always  be  done  is  a consequence  of  the  nonnegativity 
requirement  of  X and  the  boundedness  to  the  solution  set  to  (*) 
caused  by  the  fact  that  ^ The  question  of  finite  termination 

is  addressed  in  the  following  section. 


3.j>.  Conditions  for  Finite  Convergence. 

Recall  that  the  algorithm  generates  a sequence  of  m and 
(m+1) -simplexes  r^,  ,.  . One  way  to  prevent  this  sequence 

from  being  Infinite  is  to  force  it  to  remain  inside  some  compact  set, 
then  by  property  (U)  of  D.5.?,  the  sequence  must  be  finite.  The  way  to 
accomplish  this  is  to  give  conditions  on  S which  guarantee  that  outside 


of  some  compact  set  there  is  no  solution  to  (*),  and  since  each  m-simplex 
generated  by  the  algorithm  satisfies  (*),  the  path  cannot  leave  this 
compact  set.  This  is  the  essence  of 

m / ni 

Theorem  3.1.  Let  S;R  ->  (R  ) be  the  usable  point  to  set  map  whose 
fixed  point  is  sought  and  let  ^ be  a triangulation  of  r”  x [0,1] 
as  described  above.  Suppose  there  are  positive  numbers  a and  b and 
a point  x'  in  r'^  such  that  if  z B(x',a)  then  {z'-z,  y-x' ) < 0 
for  all  yS  B(z,b)\^B(x' , a)  and  z'  € S(z).  If  mesh(9>^)  <b  then 

(a)  A linear  approximate  fixed  point  of  S is  computed  in  a finite 
number  of  steps. 

Cb)  Any  linear  approximate  fixed  point  lies  in  B(x' . a+b). 

Proof.  See  Theorem  5-1  of  Merril.l  [3B].  || 

For  each  application,  the  hypotheses  of  the  theorem  are  shown 
to  hold  and  thus  finite  convergence  is  guaranteed.  All  that  remains  is 
to  take  a sequence  ■)  of  tri angulations  of  R™  x i0,l]  with  the 

property  that  mesh(9^^)  ->  0 as  k ->  ».  Now  a sequence  of  fixed 
points  will  be  generated  and  a convergent  subsequence  will  be  enough 
to  yield  a fixed  point  of  S as  is  stated  in 

Theorem  5.2.  Let  be  a sequence  of  triangulations  of  r”  X [0.1] 

as  described  above  (i.e.  mesh(^  ) -» 0 as  k ->  «).  If  there  are 

and  b together  with  a point  x'  £ r”^  and 


positive  real  numbers  a 


the  point  to  set  map  S which  satisfy  the  hypotheses  of  T.ji.l  then 

(a)  Each  linear  approximate  fixed  point  lies  in  3(x',a  + b)  and  will 

be  computed  in  a finite  number  of  steps. 

k k k 

(b)  Either  x t S(x  ) for  some  k or  any  cluster  point  of  {x  ^ 

is  a fixed  point  of  S,  where  x is  the  kth  linear  approximate 
fixed  point. 

Proof.  See  Theorem  5.2  of  Merrill  [58j.  || 

The  only  thing  left  to  develop  is  a computerized  method  for 
generating  the  sequence  ] of  triangulations  with  m.esh(yn  ) 0 

as  k ->  oo  . Various  people  have  had  needs  for  tri angulations  and  have 
developed  their  own,  e.g.  Kuhn  [32],  Scarf  and  Hansen  [25]^  Eaves  f l‘>  1 , 
Merrill  [33].  Recently  Todd  [52]  developed  a very  elegant  way  to 
generate  a sequence  of  triangulations  as  described  above.  They  are 
referred  to  as  "Union  Jack"  triangulations.  The  current  version  of  the 
computer  code  uses  this  method  and  computational  results  have  indicated 
that  it  "s  quite  efficient. 

This  completes  the  description  of  an  algorithm  which  attempts 
to  compute  fixed  points  of  usable  point  to  set  maps.  The  next  chapter 
deals  with  conditions  under  which  this  algorithm  may  be  applied  to  equality 
constrained  optimization  problems. 
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CHAPTER  '4 


THE  APPLICATION  OF  DECOMPOSABILITY  TO 
ECUALITY  CONSTRAINED  OPTIMIZATION 


^.1,  Introduction  and  Preliminaries. 


Consider  the  problem 


P:R^  -»  U {+  oo] 
where  G:  R^  ->  r"^ 

Q:R^  -»  R^  U {+  oo) 

The  objective  of  this  chapter  is  to  state  some  sufficient  conditions  on 
the  functions  P,  G and  Q such  that  (l)  P.if.l  may  be  formulated  as 
a fixed  point  problem  and  (2)  the  algorithm  described  in  Chapter  5 can 
be  used  to  solve  the  resulting  problem.  Since  global  convergence  is 
sought  one  might  expect  that  the  conditions  obtained  will  be  very  restric- 
tive. This  is  in  fact  the  case;  however,  computational  experience  is 
indicating  that  the  method  is  viable  on  a reasonable  class  of  problems 
(see  Chapter  7).  More  specifically  the  first  step  will  be  to  develop 
conditions  on  P,  G and  Q so  that  P.U.l  may  be  solved  by  finding  the 
fixed  point  of  some  usable  point  to  set  map.  The  second  step  will  be 
to  put  additional  conditions  on  these  functions  so  that  the  resulting 
point  to  set  map  will  satisfy  the  hypotheses  of  T,^.2.  This  will  ensure 
global  convergence  of  the  algorithm. 


(P.'+.l) 


min  P(z) 
s.t.  0(7)  = 0 
Q(z)  < 0 


z £ R" 


Under  the  next  two  assumptions,  Merrill  [38]  developed  a fixed 
poiiit  formulation  for  the  following  special  case  of  PJt.l 

p-.R®  U {+ 

q:R"'  R^  U {+  oo] 

Assumption  U.l.  Assume  that  p and  q are  closed  proper  convex  functions 
with  dom(q^'  = R^  for  each  i = 1,...,  i. 

Assumption  h.2.  Define  the  function  t;R°^  ->  R^  by 

t(x')  = majc{q^(x)/i  = Assume  that  {x  : R^ltCx)  < 0)  belongs 

to  int(dom  p).  (The  function  t will  have  this  interpretation  for 
the  remainder  of  the  thesis.) 

Now  one  can  define  the  point  to  set  map  S:R”  -»  (r''^)  by 

X - 8p(x)  if  t(x)  < 0 

S(x)  = • X - hull(5p(x)  U ^t(x))  if  t(x)  = 0 

X - dt(x)  if  t(x)  > 0 

The  first  thing  to  verify  is  that  S is  usable. 

Theorem  ^>.1.  If  A.i+.l  and  A.  4.2  hold  then  S is  usable. 

Proof.  See  Theorem  12.1  of  Merrill  [58].  || 
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I The  relationship  between  fixed  points  of  S and  solutions  to 

P.l.l  is  stated  in 

Theorem  U.2.  If  A. 1.1  and  A.l.P  hold  and  if  inf(t(x) 'x  t R™)  < 0 then 
x'  G S(x' ) iff  x'  solves  P.1.2. 

Proof.  See  Theorem  IP..!!  of  Merrill  [38].  || 

Unfortunately  this  theorem  does  not  guarantee  that  the  algorithm 
described  in  Chapter  3 will  compute  a fixed  point  of  S.  In  order  to 
accomplish  this  it  is  necessau'y  to  have 

Assumption  1.3.  Assume  the  function  q has  no  common  direction  of 
recession  other  than  0 (see  D.1.7). 

Now  algorithmic  convergence  is  established  in 

Theorem  1.3.  If  A.l.l-A.1.3  hold  then  the  hypotheses  of  T.3.2  are  met 
and  in  addition  if  a = inf[t(x)/x  G r''^}  then 

(a"!  The  condition  a > 0 may  be  detected  in  a finite  number  of  steps. 

(b)  The  condition  a = 0 implies  that  any  fixed  point  of  S is 

' feasible  for  P.1.2. 

(c)  The  condition  a < 0 is  a necessary  and  sufficient  condition  for  a 
fixed  point  of  S to  solve  P.1.2  and  the  algorithm  will  compute  a 
linear  approximate  fixed  point  in  a finite  number  of  steps. 

Proof.  See  Theorem  12.3  and  Corollary  12.1.1  of  Merrill  [38].  H 

lu 


The  next  logical  step  Is  to  find  so-iie  method  of  transforming  P.^.l 
into  a problem  of  the  form  P.k.".  Perhaps  the  first  thing  that  comes  to 
mind  is  to  replace  the  eoualit^  constraints  G(z'  =0  by  two  equivalent 
inequality  constraints  0(2)  < 0 and  -G(z)  < 0.  Unfortunately,  in  this 
case  part  (b)  of  T.U.i  says  that  the  algorithm  can  only  be  expected  to 
generate  a feasible  point.  Thus  it  is  necessary  to  find  another  approach. 
This  is  done  in  the  next  section. 


k.2.  Transforming  Equality  Constrained  Optimization  into  a Fixed 
Point  Problem. 

In  the  spirit  of  decomposability  the  idea  is  to  use  the  equality 
constraints  to  solve  for  some  of  the  variables  (called  the  basic  variables) 
in  terras  of  the  rest  (called  the  nonbasic  variables).  Thus  the  original 
variables  z r;  will  often  be  written  (x,y)  for  x ' r"'  and  y R*^, 

X being  the  nonbasic  variables  and  y being  the  basic  variables.  In 
order  to  show  how  and  when  this  can  be  done, define  a point  to  set  map 
H on  R*^  by  H(x)  = {y  ■-  R*^'G(x,y)  = 0).  Note  H(x'  might  be  eiupty 
so  let  X = {x  u r"*  H(x)  is  nonempty).  The  next  two  assumptions  will 
allow  the  proper  transformation  of  P.^*.!  into  P.1.2. 

Assumption  4.1.  Assume  X = R . 

Assumption  U.y.  Assume  H(x)  is  a singleton  for  each  x £ X. 

From  A.U.t'  it  is  possible  to  define  a function  h;R°'  -»  R*' 
by  h(x)  = that  unique  element  of  H(x).  Hence  0(x,h(x))  = 0 for  each 


x e R“  . 


SO 


FrcmA.ii.U  it  Is  possible  to  define  the  fonctions 
q;R"’‘->R‘'  by  p^x'  = P(x,h(x}l  and  q(x’  QAx.h  x respectively. 

This  is  the  desired  trensfonaaticn  of  P.4.1  into  P.ii..  as  is  established 
in 

Theorer.1  b.4.  If  A.h.4  and  A.'t.j  hold  and  if  h,  p and  q are  defined 
as  above  then  x - P."''  solves  P.4.2  iff  (x,h''x))  solves  P.4.1. 

Proof.  Suppose  first  that  x solves  P.4.2.  Then  (x,h(x);  is  feasible 
for  P.4.1  since  C}(x,hfx))  - C and  Q(x,h(x))  < 0.  In  order  to  show 
that  (x,h(x))  actually  solves  P.4.1  let  (x',y’;  be  any  other  feasible 
solution  to  P.4.1.  It  must  be  shown  that  p(x,h(;:))  < P(x',y').  Since 
y’  1 K{x’)  = (h(x')}  one  need  only  show  that  P(x,h(xi)  < P(x',h(x');, 
but  this  follows  from  the  optimality  of  x for  P.4.2  since  P(x,h(xi] 

= p(x)  < p(x’)  = P(x’,h(x')).  This  proves  the  necessary  part  of  the 
theorem.  For  the  sufficiency,  suppose  (x,h(K)l  solves  P.4.1.  Tlien 
X is  certainly  feasible  for  P.4.2.  In  order  to  show  that  x actua’ iy 
solves  P.4.2  let  x'  be  any  other  fea.sible  solution.  It  must  be  shewn 
that  p(x)  <p(x').  Since  (x'.h(x'i)  is  also  feasible  for  P.4.1  it 
follows  that  p(x)  - P(x,h(x1^  < P(x',h(x'))  = p(x' ) as  desired.  '1 

Now  that  the  transformation  of  P.4.1  into  P.4.?  has  been  established 
it  is  desirable  to  hs.ve  conditions  on  P,  G,  and  Q which  ensures  A.  *.l- 
A.4.b.  The  next  theorem  puts  sufficient  conditions  on  G which  makes 
A. 4. 4 and  A. 4. 5 hold.  It  is  a modification  of  the  implicit  function 


theorem  and  is  stated  as 


Theorem  ^4.3.  Let  G:R'”  x ->  R*^  be  continuously  differentiable  on 
R™  X R*^  and  suppose  there  is  a point  (a,b)  u r'"  x R*^  such  that 
(l)  G(a,b)  = 0 and  (2)  the  matrix  E = DyG(a, b)  Is  nonsingular.  Also 
suppose  there  is  a constant  0 < A < 1 such  that 

III  - E‘’‘DyG(x,y)||  < A for  all  (x,y,l  r”  x r"^  . 

Under  these  conditions  A. 1.1  and  A.k.y  hold. 

Proof.  It  will  be  shown  that  there  is  a unique  function  h:R”  -»  R*^  such 
that  G(x,h(x))  = 0 for  each  x R™.  To  this  end  define  a new  function 
L:R”  X r”  -»  r'^  by  L(x,y)  = y - E ^G(x,y).  For  each  fixed  value  of  x, 

L will  be  shown  to  be  a contraction  mapping  and  hence  will  have  a unique 
fixed  point  h(x)  C r"^.  Note  that  L(x,h(x) ) = h(x)  iff  G(x,h(x))  - 0. 

To  show  that  L is  a contraction  map  in  the  y coordinate,  the  constant  A 
in  the  hypothesis  will  be  used  to  conclude  that 

||L(x,y)  - L(x,y»)||  < A!|y-y' |l  for  all  y,  y*  c R*^  . 

So  it  is  necessary  to  bound  ||L(x.y)  - L(x,y')||.  This  is  precisely  the 
essence  of  T.1.1.  In  order  to  apply  it,  the  function  L need  only  be 
differentiable  in  the  y-coordinates  which  of  course  it  is  and  in  fact 

DyL(x,y)  = I - E‘^DyG(x,y)  . 


Upon  applying  the  bound  in  T.1.1, 


liL(x,y)  - L(x,y')l|  < 3up{  IlD^LCx^y  *■  A’ (y-y' ) ) il '0  < A'  < 1)  Hy-y' 

supdil  - e"'-  B^G(x,y+ A'{y-y’ , 'J/0  A'  < 1]  ||y-y’ i! 

1 Aily-y'ii  . 

the  last  inequality  being  justified  by  the  hypothesis.  Thus  in  fact  L 
is  a contraction  mappitig  in  the  y-coordinates  for  each  fixed  value  of 
the  x-cooidinates  and  this  completes  the  proof,  ll 

Corollary  4.1.  If  G is  linear  and  has  rank  n then  A.t.l  and  A.t.b  hold. 

Proof.  Let  G(x,y)  = Ux  + Vy  + v where  V is  an  (n  x n)  nonsingular 
matrix,  U is  an  (n  x m)  matrix  and  w is  an  n-vector.  Choose  any  A 
with  0 < A < 1 and  set  (a.b)  = (O,  -V  ^7) . Now  the  hypotheses  of 
T.B.5  hold  so  A.  u.i+  and  A.  A.  5 do  also.  jl 

Supposing  now  that  this  function  h(x)  exists,  conditions  can  be 
placed  on  P,  Q and  h so  that  A.^.l-A.^i.J  ..old. 

Proposition  U.l.  Suppose  P and  Q are  closed  proper  convex  functions 
and  that  there  exists  x',  x"  € such  that  P(x',h(x'))  < “ and 

Q(x",h(x'M)  < 00  . If  h is  linear  then  A.i+.l  holds. 

Proof.  It  must  be  shown  that  p and  q are  closed  proper  convex 
functions  with  dom(q^)  = P.'^  for  each  i = Since  h Is 

defined  on  all  of  r"''  (by  A.ft.U),  dom(q^)  = for  each  1 = I,....!'. 
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Not  p is  shown  to  be  convex.  To  do  this  let  x,  w . R®  and  A C [0,1] 
Then  the  linearity  of  h and  the  convexity  of  P it  follows  that 

p(Ax  + (l-A)w)  = p(Ax  + (l-A)v,  h(Ax  + (l-A)w)) 

= p(Ax  + (l-A)w,  Ah(x)  + (l-A)h(w)) 

P(A(x,h(x))  + (1-A)  (w,h(w))) 

< AP(x,h(x))  + (1-A)  P(w,h(w)) 

= Ap(x)  + (l-A)  p(w^  . 

That  p is  closed  and  proper  is  straightforward.  A similar  argument 
shows  q to  be  a closed  proper  convex  function  also.  |1 

In  order  to  weaken  the  assumption  that  h is  linear  it  is 
necessary  to  place  additional  structure  on  P and  Q.  This  is  done  in 

Proposition  U.2.  Suppose  P and  Q,  are  closed  proper  convex  functions 
and  that  there  exists  x',  x"  t r”'  such  that  P(x',h(x'))  < and 
Q(x",h(x’’))  < ».  Suppose  in  addition  that  P and  Q are  isotone 
(see  D. 1.8)  in  the  y-coordinates  for  each  fixed  value  of  the  x-coordinates 
If  h is  convex  then  A.U.l  holds. 

Proof.  As  in  the  previous  proposition,  dom(qj^)  - for  each  i = 1, . . . , 
Next  p is  shown  to  be  convex.  To  do  this  let  x,  w C r”  and 
AS  [0,1].  From  the  convexity  of  h, 

h(Ax  + (l-A)w)  < Ah(x)  + (l-A)  h(w)  , 
and  since  P is  isotone  in  the  y-coordinates  it  follows  that 


P(>x  + (l-A)w,  h(\x  + (l-A)w))  < P(Ax  + (l-A)v,  Ah(x)  + (1-a)w,  Ah(w)) 

= P(a(x,  h(x))  + (l-A)(v,  h(w) )) 

< AP(x,h(x))  + (l-A)  P(w,h(w)) 

the  last  inequality  being  .justified  by  the  convexity  of  P,  The  left 
most  side  of  the  inequality  is  p(Ax  + (l-A)w)  and  the  right  most  side 
is  Ap(x)  + (l-A)  p(w).  Hence  p is  convex.  That  p is  closed  and 
proper  is  straightforward.  A similar  argument  shows  q to  be  a closed 
proper  convex  function.  |1 

Proposition  If  dom(P)  = then  A,h.2  holds. 

Proof.  Obvious . || 

Proposition  4.4.  If  Q has  no  common  direction  of  recession  other  than 
0 and  if  h is  linear  then  A.h.^  holds. 

Proof.  This  is  done  by  contradiction  so  assume  d £ is  a nonzero 
common  direction  of  recession  of  q.  Then  there  is  a b £ R^  and  an 
x £ R*”  such  that  q^(x  + Ad)  < b for  all  A > 0 and  i = 

It  will  be  shown  that  (d,h(d))  is  a nonzero  common  direction  of 
recession  of  Q.  Clearly  (d,h(d))  is  nonzero  since  d is.  From  the 
linearity  of  h it  follows  that  for  all  A > 0 and  i = 

Q^((x,h(x))  + A(d,h(d)))  = Q.^{x  + Ad,h(x)  + Ah(d)) 

= Q^(x  + Ad,  h(x  + Ad)) 

= q^(x  + Ad) 

< b. 

Hence  (d,h(d))  is  the  desired  common  direction  of  recession  of  Q 
and  this  contradiction  proves  the  claim.  || 


55 


In  order  to  weaken  the  assumption  that  h is  linear  it  is  necessary 
to  place  additional  structure  on  P and  ti.  This  is  done  in 

Proposition  4,  Suppose  h is  convex.  Assume  also  that  Q is  convex 
and  isotone  in  the  y-coordinates.  If  Q has  no  common  direction  of 
recession  other  than  0 then  A.U.3  holds. 

Proof.  Again  this  will  be  done  by  contradiction,  so  assume  d € r”  is 
a nonzero  common  direction  of  recession  of  q.  Hence  there  is  a b :: 
and  X e R**^  such  that  q^(x  + Ad)  < b for  each  A > 0 and  i = 1, 

To  construct  a nonzero  common  direction  of  recession  of  Q let  W 
(an  (n  x m)  matrix)  be  a subgradient  of  h at  x (see  D.1,2).  It  will 
be  shown  that  (d,Wd)  is  a nonzero  common  direction  of  recession  of  Q. 

It  is  clearly  nonzero  since  d is.  From  the  definition  of  a subgradient, 

h(x)  + A(Wd)  < h(x  + Ad)  for  all  A > 0; 

and  since  Q is  isotone  in  the  y-coordinates  it  follows  that  for  all 
A > 0 and  i = 

Q^(x  + Ad,  h(x)  + AWd)  < Qj^(x  + Ad,  h(x  + AWd)) 

= qj^(x  + Ad) 

< b . 

Hence  (d,Wd)  is  the  desired  common  direction  of  recession  of  Q and 
this  contradiction  proves  the  claim.  || 
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In  applications  it  Is  very  often  the  case  that  neither  nor 

AJ+.5  will  hold.  Take  for  example  the  function  G:R^  ->  by 
G(x.y)  = x^  + y^  - 1.  In  this  case  X = f-1, 11  and  H(x)  = 

{ \/.-x^,  - J\  -X  ] for  each  x _ X.  One  would  like  to  be  able  to  find 
a fixed  point  formulation  of  in  which  the  resulting  point  to  set  map 

is  usable,  ITiere  are  several  problems  that  immediately  present  themselves. 
The  first  is  that  the  function  h is  no  longer  well  defined.  Recall 
that  for  each  x C r”',  h(x)  was  that  unique  element  such  that 
G(x,h(x))  =0.  In  the  current  situation  there  may  be  many  choices  for 
h(x)  (since  H(x)  need  not  be  a singleton).  Even  supposing  one  were 
able  to  construct  this  choice  function  h(x),  the  next  problem  is  that 
h is  a mapping  from  X into  R^.  If,  as  before,  one  were  to  define 
p(x)  = P(x,h(x))  and  q(x)  = Q(x,h(x))  for  each  x I X then  A.U.l 
will  not  hold  unless  X - r'”.  If  X R°^  then  the  resulting  point  to 
set  map  may  not  be  usable  (even  though  X might  be  compact  and  convex). 

No  usable  point  to  set  map  has  been  found  to  overcome  these  difficulties, 
however,  one  which  is  very  close  and  has  worked  in  practice  is  described 
in; 

Theorem  U.6.  Suppose  X is  a compact  convex  subset  of  R™  with  x'  € X. 

Suppose  further  that  A.U.5  holds  so  there  is  an  h:X  -> r"  such  that 

G(x,h(x))  = 0 for  each  x € X.  Let  p;X  R^  U {+<»)  be  defined  by 

p(x)  = P(x,h(x))  and  let  q;X  -»  R^  be  defined  by  q(x)  = Q(x,h(x)'i 

and  suppose  p and  q are  closed  proper  convex  functions  on  int(Xi, 

111  / m 

Then  the  point  to  set  map  S:R  (R  ) defined  by 
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if  X ^ int(X) 


(x'l 

X - r^p(x) 

X - hull(i'^p(x)  U <')t(x)) 

X - c)t(x) 

has  the  following  properties: 

(a)  If  x'  G int(X)  then  any  fixed  point  of  S belongs  to  int(X). 

(b)  If  x'  £ int(X)  then  any  fixed  point  of  S solves  the  problem 

rain  p(x) 

s,t.  q(x)  < 0 provided  a - inf{t(x)/x  £ int(X)}  < 0 . 
x G int(X) 

(c)  The  algorithm  when  implemented  on  S computes  either  a fixed  point 
of  S or  a point  in  bd(X). 

Proof.  Part  (a).  Let  x : S(x).  Suppose  contrary  to  the  conclusion 
of  (a)  that  X ^ int(X),  then  S(x)  - {x')  and  so  x = x'  £ int(X). 

This  contradiction  establishes  that  any  fixed  point  of  S belongs  to 

Int(X). 

Part  (b).  Let  x € S(x).  First  it  will  be  shown  that  x is 
feasible,  i.e.  q(x)  < 0 and  x G int(X).  Part  (a)  shows  that 
X G int(X)  and  by  hypothesis  a < 0 hence  q^(x)  < t(x)  < 0 for  each 
i = 1,...,  i.  Thus  X is  feasible.  If  t(x)  0 then  the  fact  that 
X is  a fixed  point  of  S yields  0 G '^p(x)  and  the  optimality  of 
follows  fix>m  the  convexity  of  p on  int(X).  If,  on  the  other  hand, 
t(x)  0 then  0 = Aa'  + (l-X)b'  for  some  a'  t Sp(x'  and  b'  G r)t(x) 


if  t(x)  < 0 

if  t(x)  =0  if  X G int(X) 
if  t(x)  > 0 
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and  A G [0,1].  Since  a < 0 and  t(x)  =0  It  follows  that  A > 0 
vdience  a'  = r(l-A)/A]h'.  To  see  that  x is  actually  optimal  let 
z e int(X)  be  any  othei  feasible  point.  It  must  be  shown  that 
p(z)  > pi^x).  By  the  fact  that  z is  feasible  and  that  b'  u bt{x) , 

0 ^ (b',z-x)  = (b',z-x)  , 

and  since  a'  £ [5p(x)  , 

p(z)  > p(x)  + (a', z-x) 

= pfx)  - [(1-A)/A]  (b',z-x)  > p(x)  . 


This  shows  that  x actually  solves  the  problem. 

Part  (c).  The  proof  will  be  done  by  showing  that  the  algorithm 

generates  a point  x t X with  the  additional  property  that  if  x ^ bd(X) 

then  X € S(x).  To  actually  construct  this  x,  consider  a sequence  of 

triangulations  of  X [0,1]  whose  mesh  is  going  to  0 for  k -»oo. 

For  each  fixed  value  of  k the  algorithm  will  generate  a seauence  of  m 

and  (m+l)-simplexes  such  that  each  m-simplex  is  completely  labelled.  .Since 

this  collection  can  never  leave  a compact  set  it  must  be  finite  and  so 

Ic  Ok  mk 

there  will  be  a completely  labelled  ra-simplex,  say  t =hull({(x  , 1) (x  , 

consequently  there  are  scalars  A^^,  ...  , a”'^  > 0 together  with  points 


s(x°^. 


mk  _ „ . mk.  v,  4. 
, z £ Svx  ) such  that 


V .ik  ik  V .,1k  ik 
,1.  A z = Z.  A X 


i=0 


1=0 


E A^^=  1 
i=0 


1))) 
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Since  the  mesh(2^  ) is  going  to  0,  it  follows  that  there  are  points 
X,  2*^,  ...  , z”“  £ R™  together  with  nonnegative  scalars  }P , ...  , a"* 


ajid  a subsequence  K such  that 


(1) 

{x  ) 

-»  X 

for 

k £ 

K 

and 

each 

i = 0,. 

. . ,m. 

(2) 

i 

for 

k £ 

K 

and 

each 

i = 0,. 

. . ,m. 

(3) 

(£*') 

for 

k £ 

K 

and 

each 

i = 0,. 

. . , m. 

First  it  will  be  shovti  that  x £ X.  If  x £f  X then  {z^^)  x’  for 
k £ K and  each  i = 0,  ...,m.  Hence  x = x'  £ int(X).  This  contradiction 
establishes  that  x £ X. 


Next  it  will  be  shown  that  if  x ^ bd(X)  then  x £ S(x).  So 
suppose  X ^ bd(X).  Then  x £ int(X)  and  hence  S(x)  is  a convex  set. 
To  show  X £ S(x)  it  will  be  shown  that  z^  £ S(x)  for  each  i 
and  thus  x = A^z^  £ S(x).  To  see  that  z^  £ S(x)  for  each 

i = 0,  ...,m  note  that  S(x)  is  usable  on  int(X)  so  by  the  upper 
semi continuity  of  S and  (l)  and  (2)  above,  z^  £ S(x)  for  each 


i = 0,...,m..  This  concludes  the  proof.  I1 


Throughout  the  entire  chapter  it  has  been  assumed  that  A.i+.5 
always  held  and  an  example  was  presented  where  A. 4. 5 did  not  hold.  It 
will  be  shown  that  there  is  a possible  method  for  dealing  with  this  type 
of  problem  but  certain  aspects  of  the  approach  render  it  computationally 
infeasible.  If  A. 4. 4 holds  then  the  function  h(x)  will  choose  a 
very  special  element  of  H(x)  in  such  a way  that  x solves  P.4.2 
iff  (x,h(x))  solves  P.4.1.  Unfortunately  no  computational  Implementation 
for  this  choice  function  has  been  fovmd.  This  function  is  described  in 


Go 


Theorem  '*.7.  Suppose  h(x)  is  compact  and  convex  for  each  x € k'*'*  and 
that  P is  continuous.  Suppose  further  that  for  each  x £ there 
is  a y £ H(x)  such  that  Q(x,y)  < 0.  Then  the  choice  function 
h:  defined  by  h(x)  - any  solution  to 

min  P(x,y) 

Q(x,y)  < 0 
y £ H(x) 

satisfies  x solves  P.^.2  iff  ^x,h(x))  solves  P.4.1. 

Proof.  Suppose  first  that  x solves  P.4.2.  Then  (x,h(x'  * is  feasible 
for  P.4.1.  Let  (x',y')  be  an„'  other  feasible  solution  for  P.4.1. 

Then  by  the  defin'  tion  of  h.  P(x',y')  >P(x',h(x'))  =p(x''  >p(x)  =P(x,h(x)) 
and  so  (x,h(x  ' actually  solves  P.4.1.  This  proves  the  necessary  part 
of  the  theorem.  For  the  sufficiency  part  suppose  (x,h(x))  solves  P.4.1. 

Then  x is  certainly  feasible  for  P.4.2.  Let  x'  be  any  other 
feasible  point  for  P.4.2.  It  must  be  shown  that  p(x' ) >p(x).  Since 
(x',h(x'))  is  feasible  for  P.4.1  it  follows  that 

p(x)  = P(x,h(x))  < P(x',  h(x'))  = P(x’)  . 11 

This  chapter  has  dealt  with  theoretical  conditions  on  the  original 
functions  P,  R and  Q which  allows  the  algorithm  described  in  Chapter  3 
to  solve  P.4.1.  It  is  now  time  to  focus  on  the  computational  aspects  of  the 
problem.  It  will  also  be  desirable  to  find  some  techniques  to  increase 
the  efficiency  of  the  algorithm.  These  are  the  topics  of  the  next  two 


chapters . 
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CHAPTER  5 


COMPUTER  CONSIDERATIONS 

^>.1.  Introduction  and  Preliminaries. 

In  the  previous  chapter  a theoretical  approach  was  suggested  for 
solving  P.it.l.  There  is  however  a considerable  gap  between  the  theoretical 
framework  and  an  actual  computer  implementation.  In  this  chapter  a 
computational  method  based  on  the  previous  theory  is  developed  in  detail. 
The  first  step  in  this  direction  involves  a careful  analysis  of  exactly 
what  quantities  must  be  computed  and  methods  are  proposed  for  accomplish- 
ing these  tasks  efficiently.  It  will  become  evident  that  the  amount  of 
work  required  is  quite  large  and  one  naturally  asks  if  there  are  ways 
to  reduce  the  computational  effort.  This  topic  however  will  be  reserved 
for  Chapter  6. 

Given  P.i4.1  recall  that  for  each  x d R™,  H(x'  = (y£R^/G(x,y)  =0} 
and  X = {x  G r”/H(x)  is  nonempty! . At  the  end  of  Chapter  !+  it  was  shown 
that  the  case  where  H(x)  was  not  a singleton  for  each  x £ X,  was  a 
very  difficult  and  seemingly  unmanageable  problem.  Therefore  for  the 
duration  of  this  chapter,  will  be  assumed.  Also  recall  that  the 

following  point  to  set  map  was  developed  to  solve  P.^4.1, 

{x']  if  X £'  int(X) 

X - t^p(x)  if  t(x!  < 0 

X - hull  (hp(x)  U dt(x))  if  t(x)  =0  if  x£  int(X) 

X - (^(x)  if  t(x)  > 0 


6? 


where  x'  £ X.  Finally,  recall  how  the  algorithm  of  Chapter  will  attempt 
to  find  a fixed  point  of  S.  It  will  use  a trianguiaiion  of  R*”  x ;0,1] 
in  which  the  vertices  lie  in  either  r'''''  x {0!  or  r”'  - {1).  The 
algorithm  starts  vvith  a special  m-s'mplex,  say,  = hull{  ( (x^,  1) , . . . , i x''^,  1 i } 
containing  the  starting  point  (w,0)  . x {0}  in  its  interior.  Con- 

sequently there  is  a basic  feasible  solution  A to 

AA  - (0, . . . 0,  1) 

A > 0 


where 


(x,u) 


(ffx;-x,  1)  for  some  f(x)  £ S(x)  if  u = 1 
(w-x, 1)  if  u = 0 . 


The  method  proceeds  to  generate  a vertex  in  the  triangulation  of  the  form 

(x,u)  with  u £ [0,1),  For  each  sucVx  point  it  is  necessary  to  compute 

A f , . This  vector  will  be  brought  into  the  current  basis  and  a 
« , X t ^ / 

dropping  vertex  will  be  unicuely  determined.  This  in  turn  will  generate 
a new  incoming  vertex  and  the  process  is  repeated.  Note  that  when  then 
incoming  vertex  (x,u)  has  u = 0,  the  computation  of  A / \ is  very 

simple,  however  when  u = 1 it  is  necessary  to  generate  a point 
ffx)  G S(x).  The  flow  chart  of  Figure  5*1  shows  the  necessary  computations 
to  accomplish  this.  The  first  three  computations  are  discussed  in  the 
next  section  and  the  other  four  computations  are  iiandled  in  the  last 


section. 


5.2.  Solving  Nonlinear  Systems  of  Equations. 


Step  I'l)  of  Figure  ‘..1  i.g  ascertaining  if  x t int(X).  Amongst 
other  things,  this  requires  determining  if  there  is  a y G such  that 
G(x,y)  = 0.  Since  this  is  attempting  to  solve  a particular  nonlinear 
system  of  n equations  in  n unknovTis  it  is  going  to  be  a difficult 
task.  Several  computational  methods  exist  for  accomplishing  this. 

In  the  proof  of  T.^.5,  for  example,  a contraction  mapping  approach  was 
designed,  however  it  is  felt  that  the  hypotheses  of  this  theorem  are  too 
strong  and  that  in  most  applications  the  function  G will  not  satisfy 
these  conditions.  A more  stable  computational  tool  was  sought.  In  the 
event  that  these  constraints  satisfy  seme  differentiability  conditions, 
Nevdion's  method  may  be  applied.  These  conditions  are  supplied  in 


Theorem  5.I.  bet  x ' X be  fixed  and  suppose  y t is  such  that 

G(x,yJ  = 0,  Suppose  G is  continuously  differentiable  \in  the  y-coor- 
dinates'  in  some  neighborhood  of  y and  that  DyG(x,y)  is  nonsingular 

at  y.  Then  there  is  a neighborhood  0 of  y such  that  for  any 

0 k 

y € 0 the  sequence  {y  ) defined  by 


k+1  k 

y = y 


- D"^G(x,y^^ 

y 


G(x,y^), 


k = 0,1,... 


converges  to  y. 

Proof.  Gee  Theorem  10.?. 2 of  Ortega  and  Rhineboldt  fUl].  || 
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Optimally  one  would  have  hoped  for  a stronger  version  of  T.5.1 
in  which  the  sequence  {y  ) converged  to  some  y £ R iff  G(x,y)  = 0, 
that  is  to  say  iff  x € X,  This  of  course  is  not  the  case  as  it  is 
possible  for  Newton's  method  to  diverge  and  yet  x £ X.  Furthermore 
there  is  no  way  of  determining  if  x € int(X),  No  way  around  these 
difficulties  is  known  so  in  practice  when  Newton's  method  fails  it  is 
assximed  that  x / int(X).  This  action  has  not  caused  any  difficulties 
in  the  test  problems  reported  in  Appendix  A.  Note  that  when  Newton's 
method  fails  it  is  easy  to  compute  f(x)  € S(x)  provided  x'  £ X is 
available.  If  such  an  x’  is  not  available  at  the  start  of  a problem, 
a Phase  I method  for  attempting  to  find  such  an  x'  exists.  The  next 
proposition  shows  how  to  set  this  up. 

Proposition  S.l.  Consider  the  problem 

n 

(P.5.1)  rain  Z z, 

i=l 

s.t.  G(x,y)  + z = 0 
z > 0 
z £ . 

Then  x'  £ X iff  there  is  a y'  £ r”  such  that  (x',y',0)  solves  P.5.1. 

Proof.  Suppose  first  that  x'  £ X.  This  means  there  is  a y'  £ R° 
such  that  G(x',y')  = 0.  Thus  (x',y',0)  is  feasible  for  P.5.1.  To 
see  that  it  is  actually  optimal  note  that  the  ob,jectlve  value  is  bounded 
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below  by  0 and  since  (x',y',0)  actually  attains  that  value  it  must  be 
optimal.  To  go  the  other  way  suppose  (x',y',0)  solves  T.j.l.  From  the 
feasibility  conditions,  rT(x',y')  = 0 and  hence  x'  X.  || 

As  a result  of  this  proposition  it  will  now  be  assumed  that  x'  £ X 
is  available  and  returning  to  Figure  5.1(2),  this  completes  the  description 
of  what  to  do  if  Newton's  method  fails  to  converge. 

Note  that  when  implementing  Newton's  method  it  is  necessary  to 
supply  an  initial  starting  point  y*^  £ from  which  to  iterate.  A good 
choice  for  y*^  will  hopefully  mean  that  fewer  iterations  are  required 
for  convergence.  On  the  other  hand  a poor  choice  ■might  lead  to  slow  con- 
vergence or  no  convergence  at  all.  Under  certain  differentiability  assump- 
tions on  G,  a tangent  plane  approximation  can  sometimes  be  used  to  generate 
a good  initial  starting  point.  This  is  the  essence  of 

Theorem  5-2.  Suppose  G is  twice  continuously  differentiable  and  that 
the  point  (a,b)  € x satisfies  G(a,b)  0 with  D G(a,b)  non- 

y 

singular  and  let  x £ r"^.  Then  there  is  a constant  e,  such  that 

l|G(x,y°)||  < e||(x,y°)  - (a,b)l|^ 

where 

y°  = b - D"^G(a,b)(D  G(a,b)(x-a))  . 

J A 

Proof.  Since  G(a,b)  = 0 and  by  the  definition  of  y^, 

G(x,y°)  = G(x,y°)  - G(a,b)  - D G(a,b)(x-a)  - D G(a,b) (y°-b) 

A y 

= G(x,y°)  - G(a,b)  - DG(a,b) (x-a,y®-b)  . 
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Set  e = sup{  i|D^G((x, + A((a,b)  - (x,y*^) ) ) ||/0  < ^ 1 1)  < o’.  The 
result  now  follows  from  T.1.2.  || 

The  reason  this  starting  point  was  chosen  was  because  all  of  the 

quantities  needed  to  generate  it  will  already  have  been  computed  by  the 

algorithm.  That  is  to  say,  at  the  moment  the  algorithm  needs  a starting 

point  for  Newton's  method  it  will  have  a point  (a,b)  with  G(a,b)  = 0 

and  it  will  have  computed  -D  ^G(a,b)  D G(a,b).  Therefore,  to  compute 

y X 

y^  all  that  needs  to  be  done  is  a matrix  multiplication  and  an  addition. 
This  concludes  the  analysis  of  Newton's  method  and  it  will  henceforth  be 
assumed  that  Newton's  method  has  converged  to  y 6 with  G(x,y)  = 0. 
According  to  Figure  5.l(^)  one  must  now  compute 

t(x)  = max{Q^(x,h(x) )/i  =1,  ...  , B]  . 

Following  this  however  it  will  be  necessary  to  generate  either  an 
a'  £ f^p(x)  (if  t(x)  < 0)  or  b'  £ f^t(x)  (if  t(x)  >0).  It  will  be 
shown  in  the  next  section  that  under  certain  conditions  these  quantities 
may  be  obtained  from  the  original  functions  P,  G and  Q. 


5.5.  Computing  Subgradients. 

In  this  section  it  will  be  shown  that  if  P,  G,  Q and  h 
satisfy  some  differentiability  conditions  it  will  be  possible  to  compute 
subgradients  of  either  p(x)  or  t(x).  First  it  will  be  shown  how  to 
compute  'v^(x)  and  Dq(x)  from  the  derivatives  of  the  initial  functions 
P,  G and  Q,  emd  they  in  turn  will  be  used  to  compute  subgradients. 
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Proposition  Suppose  P,  G,  Q and  h are  differentiable  at  some 

point  (x,h(x))  with  x G int(X).  If  in  addition  D G(x,h(x))  is  non- 

y 

singular  then  p and  q are  differentiable  at  x and 

7p(x)  = V^P(x,h(x))  + ('7yP(x,h(x) ) )'^  Dh(x) 

L'q(x)  = D^Q(x,h(x))  + DyQ(x,h(x)  ) Dh(x) 

with 

Dh(x)  = -d“^  G(x,h(x))  D G(x,h(x))  . 

J X 

\ 

Proof.  That’  p and  q are  differentiable  follows  from  the  chain  rule. 
Furthermore  the  Vp(x)  and  Dq(x)  may  be  written  explicitly  as 

Vp(x)  =V  P(x,h(x))  + (V  P(x,h(x)  ))^  Dh(x) 

X y 

Dq(x)  = D^Q(x,h(x))  + DyQ(x,h(x))  Dh(x) 

It  remains  only  to  show  that  Dh(x)  = -D  ^G(x,h(x))  D G(x,h(x)).  The 

y ^ 

function  G(x,h(x))  is  identically  0 on  intCxl  so  again  applying  the 
chain  rule  yields 

D G(x,h(x))  + D G(x,h(x))  Dh(x)  = 0 . f, 

X y 

By  hypothesis,  D G(x,h(x))  is  nonsingular,  therefore  an  explicit  formula 

y 

for  Dh(x)  is  given  by 

Dh(x)  = -D‘^G(x,h(x))  D^G(x,h(x))  . i| 
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Lemma  1.  If  p and  q are  closed  proper  convex  functions  on  int(X) 


ajid  if  p and  q are  differentiable  on  int(X)  then  Vp(x)  G Sp(x) 
and  Dq(x)  S r'q(x)  for  each  x G int(X). 

Proof.  This  is  an  immediate  consequence  of  T.l.j5.  || 

Theorem  ‘5.3.  Let  p and  q be  closed  proper  convex  functions  which  are 

differentiable  on  int(X).  Suppose  i is  such  that  q. (x)  = max{q .(x)/l< j < £} 

^ 3 

= t(x).  Then  Vq^(x)  G 5^t(x)  and  Vp(x)  G Sp(x)  . 

Proof.  Let  z € int(X).  It  will  be  shown  that  t(z)  > t(x)  + (Vq^(x),z-x) 
for  then  Vq^(x)  G r^t(x) . But  since  t(x)  = q^(x)  euid  since  q^  is 
convex, 

t(x)  + (Vqj^(x),z-x>  = q^(x)  + <Vq^(x),z-x) 

< q^(z) 

< t(z)  , 


as  desired.  The  fact  that  Vpfx'i  G Sp(x)  is  a restatement  of  L.5.1.  || 

With  Proposition  5-2  and  T.5.2  it  is  possible  to  compute  either 
an  element  of  '^p(x)  or  dt(x)  and  with  these  computations  under  control 
all  of  Figure  5.1  has  been  dealt  with. 

This  chapter  has  provided  an  implementation  of  an  algorithm  to 
solve  P.4.1.  Several  difficulties  remain  unresolved  but  computational 
tests  have  shown  this  method  to  be  quite  reliable  (see  Chapter  f). 


fO 


Specifically,  a scheme  for  computing  f(x)  £ S(x)  was  analyzed.  First 
it  is  necessary  to  attempt  to  solve  a nonlinear  system  of  equations. 

If  this  is  successful  it  is  then  necessary  to  evaluate  partial  derivatives 
and  subsequently  solve  a system  of  linear  equations.  This  clearly  requires 
a lot  of  effort.  It  is  therefore  desirable  to  find  ways  of  reducing 
this  work.  This  is  the  topic  of  Chapter  6. 
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CHAPTER  6 


ACCELERATION  TECHNIQUES 

6.1,  Introduction  and  Preliminaries. 

In  the  previous  chapter  a computational  method  was  devised  to 
solve  P.U.l.  It  was  observed  that  each  time  a point  (x,u)  £ R*^  X {1) 
was  generated  a large  effort  was  required  to  compute  f(x)  G S(x) . The 
purpose  of  this  chapter  is  to  describe  various  ways  of  improving  the 
overall  efficiency  of  the  algorithm. 

An  obvious  approach  is  to  try  to  reduce  the  amount  of  work  needed 
to  generate  f(x)  £ S(x).  One  way  of  doing  this  is  to  approximate  S by 
a sequence  {Si  of  point  to  set  maps.  Then  Instead  of  generating  fixed 
points  to  PL  approximations  of  S (as  the  algorithm  currently  does) 
one  would  generate  fixed  points  of  PL  approximations  to  S . The  idea 
is  that  it  will  be  less  expensive  to  generate  aji  element  of  S (x)  than 
one  of  S(x).  Another  possible  way  to  save  work  in  generating  f(x)  £ S(x) 
is  to  reduce  the  amount  of  work  needed  to  compute  h(x)  by  using  a 
Simplified  Newton  Method  (SNM)  (see  Ortega  and  Rhineboldt  [i^l]  for 
example)  or  perhaps  a Quasi-Newton  Method  (QNM)  (see  Murray  [39]  or 
Broyden,  Dennis  and  More  [ i+] ) . Whereas  NM  requires  a matrix  inversion 
at  each  of  its  interations,  SNM  requires  only  one  matrix  inversion  for 
the  entire  procedure.  Still  another  idea  for  saving  work  is  to  avoid 
computing  t(x)  = max{q^(x)/l  =1,...  , £]  \dilch  requires  evaluating 
all  of  the  constraints.  Instead  one  might  hope  to  be  able  to  evaluate 
the  constraints  sequentially  and  stop  as  soon  as  a violated  constraint 
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is  encountered.  This  was  observed  by  Merrill  [^8]  and  he  presented  an 
example  consisting  of  convex  functions  for  which  the  above  idea  did  not 
work.  A way  around  this  difficulty  has  been  found  and  has  been  Implemented 
in  the  computer  code. 

All  of  the  previous  ideas  have  involved  reducing  the  amotint  of 
work  needed  to  compute  f(x)  £ S(x).  Another  valuable  approach  might  be 
to  somehow  reduce  the  total  number  of  times  f(x'  £ S(x)  needs  to  be 
computed.  One  possibility  is  to  use  additional  structure  of  P.i+.l 
such  as  upper  and  lower  boundes  on  the  variables.  A triangulation  of 
the  hyperrectangle  defined  by  these  upper  and  lower  bounds  has  been 
developed.  Intuitively  this  would  appear  to  be  good  since  outside  of 
the  hyperrectangle  no  solution  can  possibly  exist.  Yet  another  possible 
way  to  reduce  the  total  number  of  times  f(x)  £ S(x)  need  be  computed 
is  suggested  by  Saigal  [46].  His  analysis  can  only  be  applied  to 
P.2.1  in  which  there  are  no  inequality  constraints  and  requires  some 
differentiability  conditions  on  the  functions.  In  this  case  the  rate 
at  which  the  mesh  of  the  triangulation  goes  to  zero  can  be  greatly 
increased.  Each  of  these  ideas  is  now  made  mathematically  concrete. 


6.2.  Approximation  Techniques. 

Consider  the  problem  of  finding  a fixed  point  of  a function 
f:  R™  -»  r”.  In  many  applications  the  evaluation  of  f can  be  very  -,ime 
consuming.  It  would  therefore  seem  reasonable  to  attempt  to  reduce  the 
amount  of  work  needed  to  evaluate  f.  This  will  be  accomplished  by 
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replacing  f by  a sequence  of  functions  {f  ).  Under  certain  circumstances 
the  algorithm  of  Chapter  3 may  be  used  to  compute  a linear  approximate 
fixed  point  of  f in  a finite  number  of  steps,  and  if  the  sequence  of 
points  thus  generated  has  a cluster  point  then  this  can  be  shown  to  be 
a fixed  point  of  f.  The  motivation  for  this  approach  lies  in  the  fact 
that  each  is  easier  to  evaluate  than  the  original  f.  Tiie  circumstances 

under  which  this  approach  will  work  is  now  developed.  It  is  necessary  to 
introduce 

Definition  6.1.  A sequence  [f  } of  functions  is  said  to  be  weakly  equi- 

continuous  iff  for  each  .>0  there  is  a t > 0 and  an  integer  N > 0 

such  that  llx-zll  < & implies  ||f^(x)  - f^(z)!l  < ^ for  all  k > N. 

The  next  two  lemmas  will  be  used  in  the  proof  of  the  main  theorem; 

k k 

Lemma  6.1.  If  {f  ) converges  pointwise  to  f and  if  {f  ) is  weakly 

equi continuous  then  for  any  sequence  of  points  {x  ) converging  to 

X £ r'",  the  sequence  ')]  converges  to  f(x). 

Proof.  Let  e > 0.  By  D.6.1  there  is  a 6 > 0 and  an  integer  > 0 
such  that  ||x-yll  < 6 implies  !lf'^(x)  - f^(y)  ||  < e/2  for  all  k > N^. 

Since  {x  ) converges  to  x one  may  choose  an  integer  > 0 such 
that  ||x^  - x||  < 8 for  all  k N^.  Also  since  (f^}  converges  point- 
wise  to  f there  is  an  integer  > 0 such  that  ||f^(x)  - f(x)||  < e ^2 
for  all  k > Nj,  Set  N = max(N^, N^) . Then  for  any  k > N it  follows 
that 


7'* 


Ilf'^Cx^)  - f(x)ll  < Ilf^(x‘')  - f^(x)|l  + l|f^(x)  - f(x)|| 
< fc-/2  + e/2 


= G . 


The  first  term  is  made  small  by  the  weak  equicontinuity  of 
the  second  term  is  made  small  by  the  pointwise  convergence 
to  f.  II 


{ f^ ) and 
of  [f^) 


Lemma  6.2.  Let  [T/in  be  a sequence  of  triangulations  of  k such  that 

mesh  (?f)  -^0  as  k -)  00.  Suppose  further  that  {f^)  is  a weakly  equi- 

kL 

continuous  sequence  of  functions.  Let  f be  the  PL  approximation  to 
f^  induced  by  0^^  (see  Chapter  3).  Then  for  any  sequence  of  points 
{x  ) and  € > 0 there  is  an  integer  N > 0 such  that 
llf^(x^)  - f^(x^)  II  < e for  all  k > N. 


Proof.  The  proof  is  done  by  showing  that  for  large  enough  N,  the 

k k 

vertices  of  the  m-simplex  containing  x are  sufficiently  close  to  x 

to  apply  the  properties  of  weak,  equicontinuity.  Formally  then  let  t > 0 

be  given.  Let  = hull({x^^,  ...  , x'°^} ) be  an  m-simplex  in 

containing  x^.  Hence  there  are  nonnegative  multipliers  1?^,  ...  , a"* 

■vrtiich  sum  to  1 such  that  x^  = A'^^x^^.  Let  6 > 0 euid  N,  be  chosen 

J=0  1 

If 

by  the  definition  of  weak  equicontinuity  of  {f  !•  Choose  such 

that  llx^^  - x'^ll  < 6 for  each  j = 0,  ...,m  and  k > N^.  This  may  be 
done  since  mesh(^^)  -» 0 as  k -»  ».  Setting  N = max(N^,N.,)  it  follows 
that  for  all  k > N, 
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r 


tlf^(x'^)  - f^(x‘^)|l  = Ilf^(  Z - f^(x^)|| 

J=« 

= II  Z A‘’^(f^(x*^^)  - f^(x^))|| 
J=0 


< Z A^^||f^(xJ^  - f^(x^)|| 

0=0 


1 ^ 


^Ok 


€ 


Theorem  6. 1.  Let  { f ) be  a weakly  equicontinuous  sequence  of  functions 

which  converges  I'Ointwise  to  f.  Let  ) be  a sequence  of  triangulation 

of  r’^  such  that  raesh(^^)  -^0  for  k ->  ».  Also  let  f^  be  the 

PL  approximation  to  induced  by  If  {x^}  is  a sequence  of  fixed 

kL  k 

points  of  f such  that  (x  ] -» x for  k £ K (some  subsequence),  then 
X is  a fixed  point  of  f. 


Proof.  It  will  be  shown  that  ||f(x)-x||  =0  so  let  t > 0.  Then  for 
each  k £ K, 

||f(x)-x||  < ||f(x)-f^(x^)||  + llf^(x^)  - f^(x^)||  + ||f^(x^-x^||  + ||x^-x||  . 

Epch  of  these  terms  can  be  made  less  than  e/k  for  sufficiently  large  k. 

The  first  term  can  because  of  L,6.1  as  can  the  second  by  L.6.?.  The 

k kL 

third  term  is  0 because  x is  the  fixed  point  of  f for  each  k £ K. 

Ip 

Finally,  the  fourth  term  can  be  made  less  than  e/U  since  (x  ) -»  x 


for  k £ K by  } 'pothesis.  The  result  now  follows  by  letting  t -♦  0.  || 

J( 


The  next  proposition  shows  that  under  certain  circumstances  a 
sequence  of  PL  approximations  to  a uniformly  continuous  function  can 
generate  a weakly  equicontinuous  sequence  of  functions. 


Proposition  6,1.  Let  f : r'''  be  uniformly  continuous  and  let 

be  a sequence  of  triangulations  of  r”^‘  such  that  mesh^T^^)  0 
kL 

for  k ->  «.  Also  let  f be  the  i‘L  approximation  to  f induced  by 
then  (f^)  is  an  equicontinuous  sequence  of  functions. 


Proof.  Let  e > 0.  First  choose  5 > 0 such  that  llx-yl|  < & implies 

||f  (x'' -f  fy)  II  < c/3  by  the  uniform  continuity  of  f.  Next  choose  N such 

that  mesh(^^)  < 6 for  all  k > N.  It  will  be  shown  that  3 and  N 

satisfy  the  definition  of  equicontinuity . To  see  this, let  k > N and 

X,  y £ r"^  with  ||x-y||  < 6.  Also  let  hull({x'^^,  ...  , x°*)), 

hull((y*^^,  ...  , y'"^M  be  simplexes  in  '7^^  containing  x and  y 

0^  rtii^ 

respectively.  Hence  there  are  nonnegative  multipliers  A , . . . , A 
and  u*^^,  ...  , u"^  which  s’om  to  1 such  that  x = A^^x^^  and 

y = u^^y^^  . Consequently 

||f^(x)-f^(y)||  < ||f^(x)-f(x)||  + |!f(x)-f(y)||  + ||f (y)-f^(yMI 

= II  Z A^^(f(x^'^)-f(x))||  + ||f(x)-f(y)||  + II  ^ u^^(f(y)-f(y^^ 
1=0  i^O 

m,,.,  ra., 

< Z A^^||ffx^^)-f(x)||  + ||f(x)-f(y.||+  u^*'||f(y^-f(y^h)ll 

i=0  i=0 


e . 
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In  particular,  if  f is  a continuous  function  from  a compact  set  into 
itself  then  f will  be  uniformly  continuous  and  consequently  a sequence 
of  PL  approximations  to  f will  yield  a weakly  equicontinuous  sequence 
of  functions. 

A theorem  similar  to  T.6.1  for  the  point  to  set  map  case  can  be 
developed.  For  this  it  is  necessary  to  have 

Definition  6.2.  Let  S^:R'^  ->  (r"')  be  a nonempty  point  to  set  map  for 
each  k = 1, ?, ...  . Then  {Si  is  said  to  converge  equicontinuous ly 
to  the  nonempty  point  to  set  map  S if  whenever 

(1)  {x^}  -»  X, 

(2)  £ S^(x^)  for  all  k = 1,2,...,  and 

(5)  {z^}  -»  z then  z £ S(x). 

Theorem  ^.2.  Suppose  {S  ] is  a sequence  of  nonempty  point  to  set  maps 

which  converges  equicontinuously  to  a usable  point  to  set  map  S.  Also 

suppose  there  is  a compact  set  C with  S^(r'’^'  c C.  Let  be 

a sequence  of  triangulations  of  r'’^  with  meshC"^^)  -^0  for  k « 

k yi  k k 

and  let  f be  the  PL  approximation  to  S induced  by  m with  x 

a fixed  point  of  f . If  x is  a cluster  point  of  (x*^)  then  x £ S(x), 

Proof.  The  proof  is  done  by  showing  that  x may  be  expressed  as  a convex 

combination  of  points  in  S(x).  The  result  will  then  follow  since  S(x) 

k kL 

• rorvex.  To  begin  with,  x is  a fixed  point  of  f . Let 

be  an  m-simplex  containing  x^.  Hence 

-'“nnegat  1 V"  mult  ip  1 ler.s  }P^,  ...  , which  sum  to  1 


«>jii  iic  ik  Ic  k IcT 

such  that  Kj  A X = x . To  say  that  x is  a fixed  point  of  f 


"i=0 

, Ok  £,k,  0k>  mk 

means  that  there  are  points  z c S (x  z 


„k/  mk.  _ , 

S (x  ) for  which 


m 

>: 

i=0 


v ik  i k 
A X 


m 


- Z A 
i-0 


ik  ik 


One  would  like  to  be  able  to  take  limits  in  this  equation  therefore  a sub- 

xk  3 k i-k 

sequence  K will  be  found  for  which  {z  ],  {x  ] all  converge 

for  k £ K.  To  do  this  it  will  be  shown  that  {x^^),  and  { A^^) 

xk 

each  lie  in  different  but  compact  sets.  The  compact  set  containing  (z  } 

ik 

is  C by  hypothesis.  The  sequence  {x  ] actually  converges  to  x 

for  each  i =0,,...  ra.  This  is  because  mesh('^^)  -» 0 for  k -»  «. 
ik 

Finally  the  {A  I lie  inside  a simplex.  Hence  there  is  a subsequence 

K along  which  (z^^l  z^  for  k £ K,  {x^^}  x for  k G K,  and 

ik  i 

{A  ] -> A for  k £ K and  this  is  true  for  each  i - 0,,,.,  m.  How 

one  may  take  limits  in  the  above  equation  to  yield 

m . m . . 

Z A X = Z A z^ 


i=0 


i=0 


or  equivalently 

m . . 

X = E A z^ 
i=0 

Hence  x is  a convex  combination  of  •p , ....  z''^.  All  that  remains  to 
be  shown  is  that  z^  £ S(x)  for  each  i =0,...,  m,  and  this  follows 
from  D.6.2  since  for  each  i, 

(1)  {x^h  X for  k G K, 

(2)  z^*"  £ S^Cx^^')  for  k £ K, 

(3)  -*  P for  k C K, 

therefore  z^  £ S(x)  for  each  1 - 11 
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This  concludes  the  approximation  section,  however,  one  word  of 
caution  is  in  order  when  ijnplementing  the  idea.  If  the  early  approximations 
are  not  good  the  algorithm  might  become  "trapped"  in  the  wrong  region  and 
require  a lot  of  additional  work  to  get  back  to  the  correct  answer,  thus 
negating  the  savings.  Hence  one  may  wish  to  think  of  this  as  a "tail  end" 
procedure  depending  of  course  on  the  specific  approximation. 


6.3.  Various  Modified  Mewton  Methods. 

In  this  section  the  Simplified  Newton  Method  (SNM)  and  the  Quasi- 
Newton  Methods  (QNTi)  are  described  as  possible  alternatives  to  using 
Newton's  Method  (NM)  for  solving  nonlinear  systems  of  equations. 

Recall  how  fJM  will  work.  For  a given  point  x £ R®  it  will  take 

0 It 

a starting  point  y and  generate  the  sequence  {y  } defined  by 


= y*^  - D”^G(x,y^'  G(x,y^)  for  k = 0,1,...  . 


-1  k 

Notice  that  it  is  necessary  to  recompute  D G(x,y  ) at  each  iteration. 

y 

The  idea  behind  SNM  is  to  compute  the  matrix  W = D ^G(x, y^)  once  and 
for  all  and  then  to  generate  the  sequence  {y  } defined  by 


y = y - WG(x,y  ) for  k = 0,1, . . . . 


Figure  ^.1  shows  the  difference  between  these  two  methods  in  the  1- 
dimensional  case.  The  next  theorem  gives  conditions  under  vrtiich  SNM 
may  be  expected  to  converge. 


8o 


Figure  6. 1 


I 
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Theorem  6. 3.  Suppose  G:R"'  x r'^  ->  is  continuously  differentiable  in 

the  y-coordinates  for  each  fiKed  value  of  the  x-coordinatea.  Suppose 
also  that  (x,y)  t R*"  x r'^  with  G(x,y)  = 0,  and  det(D^G(x,y) ) ^ 0, 
Then  there  is  a neighborhood  0 of  y such  that  for  all  y®  t 0 the 
sequence  {y  ] defined  by 

y^^^  = y^  - D~^G(x,y*^)  G(x,y^),  k = 0,1, ... 


converges  to  y. 

Proof.  The  proof  is  done,  by  showing  that  there  is  a neighborhood  0 

of  y such  that  for  each  y^  £ 0 the  function  L:R”’^  x c|(0)  -+c|(0) 

defined  by  L(x, z)  = z - Dy^G(x,y^)  G(x, z)  is  a contraction  mapping 

and  hence  has  a unique  fixed  point  namely  y.  Consequently,  since 
k k 

y = L(x,y  ),  it  must  be  that  {y  ) -» y.  So  ail  that  needs  to  be 
demonstrated  is  the  existence  of  this  neighborhood  0.  The  first  property 
that  0 must  satisfy  is  that  L must  be  a contraction  mapping  on  c|(0). 
Thus  it  will  be  necessary  to  show  that  there  is  a constant  0 < c < 1 
such  that 

||l(x,  z)  - L(x,z')||  < c||z-z'||  for  all  z,  z'  £ 0. 

Therefore  it  is  necessary  to  bound  ||l(x,  z)  - L(x,z’)||.  Since  L is 
differentiable  in  the  y-coordinates,  T. 1.1, may  be  used.  This  bound 
specifies  that 
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||l(x,  z) -L(x,  z' ) II  < sup{  Id  L(x,  z+A(z' -z) ) II/a  e fO,  1] } ||z-z' ||.  Hence  0 

«y 

will  be  chosen  so  that 

sup{||D  L(x,  z + A(z’-z)  )|j/A  G L0,l]1  < c 

for  some  c between  0 and  1.  From  the  definition  of  L, 

|DyL(x,z)|  --  |l  - D"^G(x,y^)DyG(x,z)| 

< liDy'^G(x,y)  DyG(x,y)  - D"^G(x,y°)  DyG(x,y)|| 

l|D"^G(x,y°)  D G(x,y)  - D“^G(x,y°)  D G(x,z)|| 

«y  O'  V 

< i|DyG(x,y)|  |D'^G(x,y)  - D'^G(x,y°)| 

+ |D"^G(x,y°)|i  !|DyG(x,y)  - DyG(x,z)i|  . 

The  set  0 will  be  chosen  to  make  each  of  these  final  two  terms  ^ c/2, 
and  it  will  follow  that 

sup(l!DyL(x,  z+  A(  -.’-z) ) 11/A  G [0,1]}  < c . 

The  first  terra  can  be  made  less  than  c/2  by  using  the  continuity  of 
DyG(x,y)  to  choose  a > 0 such  that 

|Dy^G(x,y)  - D’^G(x,y°)||  < c/(2||DyG(x,y)  (|) 

whenever  ||y-y*^||  < Also  a t)e  chosen  together  with 

a constant  e such  that 
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||D"^G(x,y°)||  < e 

whenever  ||y-y^||  < 6^.  Finally  there  is  a < 6^^  such  that 

l|DyG(x,y)  - DyG(x,z)||  < c/(2e) 

whenever  ||2-y|l  < Defining  0 = B(y,62)  does  the  trick.  All  that 

remains  to  he  verified  is  that  L in  fact  maps  R®  x c|(0)  into  c|(0). 

To  this  end  let  z £ c((0).  It  msy  be  shown  that  ||l(x, z)  - y||  < S^. 

By  T.1.1  and  the  construction  of  0, 

||L(x,y)-z||  = ||l(x,z)  - L(x,y)|| 

< sup{||DyL(x,  z + A(y-z))||/AG  [0,1]}  l|z-y|( 

< c||z-yl| 

< llz-yll 

< V 

This  completes  the  proof,  || 

Other  variants  of  NM  are  the  Quasi-Newton  Methods  (QHM) . Basically 

these  methods  differ  from  Newton' s Method  in  that  they  do  not  require 

•1  k 

a direct  computation  of  D G(x,y  ).  Instead  this  information  is 

y 

approximated  by  local  data.  The  approximation  is  continually  updated 
throughout  the  solution  procedure.  Davidon  [10]  and  Fletcher  and  Powell 
[l8]  were  amongst  the  first  to  study  this  approach.  Only  the  (n+1) -point 
sequential  secant  method  predated  their  work. 

8U 


6 . i+ . Use  of  Upper  and  Lower  Bounds. 


In  this  section  a different  point  to  set  mapping  is  designed  to 
solve  P.^.2.  The  difference  is  that  in  the  new  fomulation  it  will  not 
be  necessary  to  compute  t(x;  = max{q.  (x)  /l  < i ■2] . Instead  it  will 
be  possible  to  check  eacli  constraint  for  feasibility  and  stop  as  soon 
as  the  first  infeasible  constraint  is  detected.  To  do  this  it  will  be 
necessary  to  assume  A.l^.l.  Tlien  it  is  possible  to  define  the  following 
collection  of  convex  ftinctions  t^rR”^  ^ by  t^(x)  - max{q  . (x)/l < j < i} 

for  each  i = i, , ^ i?,  and  the  following  usa'cle  point  to  set  maps 

T^rR*’^  ->  (r”^)  by  T^  (x)  - x - ot^(x,;  for  each  i = 1,...,  Letting 
S*;R  -»  (R  ) by  3 (x)  = x - ip(x)  then  one  may  inductively  define 

S^"^:r™  for  1 = i,  i!-l,  1 by 


j S^(x) 

if 

t^(x) 

< 0 

S^"^(x')  = I hall(S^(x)  UT^(x') 

if 

t^(x) 

= 0 

! T^(x> 

if 

t^(x) 

> 0 . 

S is  the  point  to  set  map  whose  fixed  point  will  be  shown  to  solve 
'P.k.2.  First  however  it  is  necessary  to  establish  that  is  usable 

Theorem  6. U.  If  A.^+.L  holds  and  if  {x/t'^(x)  < 0l  c int(dom  p)  then 
is  usable. 

£ j -1 

Proof.  S is  usable  by  T.l.^.  By  T.1.5  one  can  conclude  that  S 
is  usable.  Iterating  backwards  finally  yields  is  usable.  || 
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The  relationship  between  fixed  points  of 
P.U.2  is  established  in 


and  solutions  to 


Theorem  6.5.  If  u = inf{t;^(z)/z.  G < 0 then  x S^(x)  iff  x 
solves  P.^.2. 

Proof.  Assume  first  that  x t S^(x).  It  is  necessary  to  show  that 
(a)  X is  feasible  and  (b")  if  z is  any  other  feasible  point  then 
p(z)  > p(x).  The  feasibility  of  x will  be  established  by  contradiction, 
so  suppose  X G S*^(x'  is  infeasible.  It  will  be  shown  that  t^'(z)  > 0 
for  all  z and  this  will  contradict  the  fact  that  u < 0.  Since 
x £ S*^(x)  there  is  a £ [0,1],  x^  £ S^(x),  £ T^(x)  such  that 

,11^/,  ,1,  1 

X = A X + (1-A  )z  . 


Since  z^  £ T^Cx)  there  is  a b^  £ dt^(x)  with  z^  = x-b^,  hence, 

X = aV  + (l-A^)(x-b^)  . 

Note  that  if  t^(x)  > 0 then  A^  = 0 and  if  t^(x)  < 0 then  A^  = 1. 

Since  x^  € S^(x)  there  is  a A^  £ [0,1],  x^  £ S^(x),  z^  £ T‘^(x)  such 

that 

1 ^22.,,  ^2.  2 

x = A X + (1-A  )z  . 

Since  z^  £ T^(x)  there  is  a b^  £ [^t‘‘(x)  with  = x-b^,  hence 

x^  = + d-A^Ux-b'^). 


86 


Substituting  this  into  the  expression  for  x yields 


-1,2  2 

X = A A X + 


A (1-A")3 


A^(l-A^)b^  + 


(l-A^)x  - (l-A^)b^ 


Note  again  that  If  t‘"(x)  > 0 then  A^  = 0 and  if  t^(x)  < 0 then 
A^  = 1.  Continuing  in  this  manner  yields  that  for  each  i = 1, . , . , Jg, 
there  is  a A^  € [0,1],  b^  £ (>t^(x)  together  with  a c £ ^p(x)  such  that 


X = A'(x-c)  + (1-A''x  - Z e^b^ 

i=l 

where 

A'  = A^-A^ A^  and  e^  = (l-A^)(  Z A^)  . 

j=l 

(it  is  understood  that  e^  = (l-A^).)  Simplifying  this  one  has 

£ . . 

(a)  A'c  + Z eS^  = 0 

i=l 

and  as  noted  above 

(b)  t^(x)  >0  implies  A^  = 0,  for  each  i - and 

(c)  t^(x)  < 0 implies  A^  = 1,  for  each  i = 

Recall  that  x is  assumed  to  be  Infeasible  hence  for  some  k,  t (x)  > 0 
and  consequently  by  (b) , A'  = 0.  Thus  (a)  reduces  to 
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J 


£ i i 

(a')  ® b =0-  To  establish  the  desired  contradiction  it  Is 

necessary  to  use  the  convexity  of  t^  and  the  fact  that 
b^  6 5t^(x)  so 

t^(z)  > t^(x)  + (b^,  z-x)  for  all  7,  u r”  . 

Multiplying  both  sides  of  this  by  the  nonnegative  number  e^  and  summing 
over  i yields 

(d)  (2)  ^ ^i=l  z-x)  for  all  z t R™ 

= ^1=1  , 

The  last  equality  being  justified  by  (a').  Note  that  t^(x)  is  non- 
decreasing in  i so 

(e)  e^t^(z)  < ^^(z)  for  all  z e r“  . 

Also  note  that 

(f)  e^^(x)  > 0 

because  e^  > 0 and  if  t^(x)  < 0 then  by  (c)  e^  = 0.  Combining 
(e)  and  (f)  into  (d)  yields 

(Z^_-j^  e^)  t^(z)  >0  for  each  z £ r'“. 
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Finally  observe  that  =-  (l-A' ) + A'  = 1 and  hence  (z)  > 0 

for  each  z € R*".  This  is  the  long  awaited  for  contradiction.  Thus  far 
it  has  been  shown  that  x is  feasible  and  consequently  that  A'  > 0. 

Now  let  z £ R*”  be  any  other  feasible  point  for  P.ii.?.  It  must  be 
shown  that  p(z)  > p(x).  This  will  follow  from  the  convexity  of  p 
and  the  fact  that  c £ fip(x)  since 

p(z^  > p(x)  + (c,  z-x)  . 


Hence  all  that  needs  to  be  shown  is  that  (c,  z-x)  >0,  and  this  will 
be  derived  from  the  feasibility  of  z and  the  convexity  of  t^. 

Since  £ cit^(x)  , 

(g)  t^’fz)  > t^(x)  *-  (b^,  z-x)  for  each  i = 1,...,  i. 


Multiplying  both  sides  of  (g)  by  the  nonnegative  scalars  e^,''A'  and 
summing  over  i yields 


(h) 


(eVA-)  t"(z)  >Ej^^(e\/A’)  t^x)  + (Ef^^(e\'A^)b. , z-x) 


and  from  (a), 


(h')  Ef_^(e^/A')  t^(z)  > Ef_^(e^ ''A' ) t^(x)  - (c,  z-x)  . 

On  the  right  side  of  (h’)  note  that  E^_^(e^/A'^  t^(x^  = 0 since  x is 
feasible  and  since  t^(x)  < 0 implies  e^  = 0 by  (c) . On  the  left 
side  of  (h’)  note  that  'A' ) t^(z)  < )"^_j^(e^/A' ' t^(z)  by  the 

monotonicity  of  t^  in  1.  Combining  these  two  into  (h')  yields 


89 


£ 

Since  z is  a feasible  point,  t (z)  < 0 and  hence  (c,  z-x)  >0  as 
desired.  This  proves  the  necessary  part  of  the  theorem.  To  establish 
the  sufficiency,  assume  x solves  P.U.2.  It  will  be  shown  that 
X € ^(x)  and  an  argument  is  given  to  show  that  x t S^"‘^(x).  This 

argument  may  be  repeated  to  establish  that  x € s'^(x).  Note  that 
X u S^  ^(x)  by  T.l.>.  To  see  that  x G ^(x)  consider 

Case  1:  t^"^(x)  < 0 . 

In  this  case,  S’^~'^(x)  = S^”^(x)  and  therefore  x G S^"‘'(x). 

Case  2:  t'^~^(x)  ---  0 . 

In  this  case  3^  ^(x)  = hullfs^  ^(x)  (J  T'^”’^(x)  ■! . But  x G S'*^"^(x)  so 
in  particular,  x G S (x).  Since  x is  feasible  for  P.4.2  these  are 
the  only  two  possible  cases  and  that  establishes  the  proof.  1| 

At  this  point  one  would  hope  to  be  able  to  apply  the  algoritlim 
of  Chapter  3 to  this  point  to  set  map.  Unfortunately  there  is  no  guarantee 
that  a linear  approximate  fixed  point  of  will  be  computed  in  a finite 

number  of  steps.  Merrill  F 3^]  observed  this  difficulty  and  he  constructed 
an  example  where  the  constraints  were  all  convex  as  was  the  objective 
function  yet  the  algorithm  failed  because  it  did  not  find  a linear 
approximate  fixed  point  in  a finite  number  of  steps.  A way  around  this 
difficulty  is  to  ensure  that  the  algorithm  never  searches  for  a fixed 


point  of  the  PL  approximation  outside  of  some  compact  set.  A very  natural 
choice  would  he  the  hyper rectangle  defined  hy  the  upper  and  lower  bounds 
since  no  solution  to  P.4.J  can  lie  outside  this  region  anyway.  The 
only  problem  is  that  P.1.2  may  have  some  variables  which  have  no  bounds. 

In  this  case  it  is  necessary  to  impose  arbitrary  upper  and  lower  bounds. 
Each  time  a new  linear  approximate  fixed  point  is  computed,  the  artificial 
bounds  will  be  expanded.  This  is  the  essence  of 

k Ic  m 

Theorem  6, Let  {u  } and  {v  ] be  sequences  of  vectors  in  R™ 

k k k 

with  u < V and  {u  ] -»  (-  to,  - <»,  ...  , - oo)  and 

[v  1 -^  (+  TO,  + TO,  . . . , + «'  as  k ->  TO  . Define  a sequence  of  problems 

P^:  min  p(x) 

s.t.  q(x)  < 0 

u^  < X £ v^ 

X L r“ 

Then  x solves  P.l.P  iff  there  is  an  N such  that  x solves  P for 
k > N. 

Proof.  Suppose  first  that  x solves  P.1.2.  Choose  N such  that  k > N 

k k k 

implies  u < x ^ v . If  z is  any  other  feasible  point  of  P for 

k > N,  then  p(x)  > p(x)  since  x solves  P.l.i.  This  establishes  the 

Jf 

necessary  part  of  the  theorem.  Suppose  now  that  x solves  P foi 
all  k > N.  It  must  be  shown  that  x solves  P.1.2.  Clearly  x is 
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feasible  for  P.4.2  since  q(x)  <0.  If  z is  any  other  feasible  point 
for  P.4.2,  then  choose  k > N such  that  z is  feasible  for  P^.  From 
the  fact  that  x solves  P , p(x)  < p(z).  This  establishes  that  x is 
optimal  for  P.4.2.  || 

An  additional  advantage  to  using  a hyperrectangle  is  that  it  may 
be  easily  triangulated.  The  philosophy  here  is  that  "if  there  is  no 
reason  to  search  outside  a given  region  then  don't."  Also  one  would  like 
the  ability  to  control  the  "size"  of  these  simplexes.  To  show  precisely 
how  this  is  done  let  u,  v € with  u strictly  less  than  v in  all 
coordinates  (u  will  be  the  lower  bounds  and  v the  upper  bounds) . 

Also  let  . . . , n”  be  positive  integers.  Set  M = X™_^[0,N^]  and 

C = X?_^[u^,v^].  The  idea  is  to  create  a linear  homeomorphism,  L, 
from  M into  C.  Then  any  triangulation  of  M yields  one  of  C by 
applying  L to  each  simplex.  Michael  Todd  [52]  has  developed  a tri- 
angulation of  r”  which  when  restricted  to  M yields  a triemgulation 
of  M.  This  combined  with  the  mapping  L will  provide  the  desired  tri- 
angulation of  C.  It  remains  only  to  specify  the  mapping  L»M  -+0, 
so  let  (ij^,  ...  , i^)  e M then  define 

1(1^,  ...  , i„)  i„) 

where 


02 


0 


0 


r 


0 


if 


(v  -n  ) 
m m 


N™ 


This  has  been  iraplemented  into  the  computer  code  and  choices  for 
N^,  ...,  have  been  left  to  the  user's  discretion. 


6.5.  Other  Methods  and  Future  Research. 

One  of  the  more  effective  acceleration  techniques  is  that  of 
Saigal  [^^,45].  It  applies  to  the  special  case  of  P.4.2  in  which  there 
are  no  inequality  constraints,  that  is  to  say,  unconstrained  optimization. 
Furthermore  the  hypotheses  require  that  p be  a strongly  convex  twice 
continuously  differentiable  function.  In  this  case  he  was  able  to  obtain 
an  increased  convergence  rate  by  increasing  the  rate  at  which  the  mesh 
of  the  triangulations  goes  to  zero. 

Since  the  area  of  acceleration  techniques  appears  to  be  the  most 
important,  several  unsolved  problems  will  be  presented  in  the  hopes  that 
if  they  con  be  solved,  further  improvements  will  be  obtained. 

For  example,  in  Chapter  5,  it  was  mentioned  that  various 
researchers  have  developed  computer! zable  triangulations  of  R™  and 
of  the  unit  simplex,  and  with  the  results  of  section  6.4  it  is  now  possible 
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to  triangulate  a hyperrectangle.  The  next  generalization  would  be  to 
develop  such  a triangulation  of  a compact  polyhedral  set.  This  would 
enable  one  to  solve  a linear  programming  problem  in  a finite  number  of 
steps.  Another  interesting  possibility  would  be  to  find  a sequence 
of  linear  transformations,  which,  when  applied  to  the  original  tri- 
angulation of  R™  yields  a new  triangulation  of  r'”'^  with  perhaps  some 
beneficial  new  properties. 

Another  area  which  could  use  some  work  is  the  way  in  which  the 
basic  and  nonbasic  variables  are  chosen.  This  problem  has  not  been  dealt 
with  here  and  it  can  only  be  said  that  in  the  test  problems  of  Chapter  7 
a proper  choice  was  always  evident.  In  general  one  will  not  be  that 
lucky  and  it  will  be  necessary  to  find  a method  for  doing  this.  The 
choice  can  be  critical  in  the  success  of  the  algorithm.  The  following 
example  shows  that  a poor  choice  of  the  basic  and  nonbasic  variables 
can  cause  the  algorithm  to  fail. 

Example  6.1.  Consider  the  problem 

min  y - X 
s . t . y - x^  = 0 
x,y  6 R^  . 

The  solution  to  the  problem  is  x = l/2,  y = l/h  and  the  algorithm  will 
compute  this  answer  provided  the  x variable  is  chosen  as  the  nonbasic 
variable,  vrtiereas  if  y is  so  chosen,  the  algorithm  can  be  caused  to 
converge  to  x = y = 0 by  choosing  an  initial  point  x'  * -1,  y'  = 1 
with  G(x' ,y' ) =0. 
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CHAPTEP.  7 


COI-DVTER  RESULTS 


7.1.  Designing  the  Test. 

All  of  the  previous  work  has  been  used  to  develop  an  algorithm 
for  solving  P.^.l  under  rather  hypothetical  conditions.  Although  con- 
vergence has  been  theoretically  established  in  Chapter  ^4  the  ultimate 
value  of  this  (or  for  that  matter  any  other)  approach  can  only  be 
determined  by  its  actual  performajice.  Therefore  seventeen  test  problems 
have  been  solved  and  the  results  are  compared  against  those  obtained  from 
GEG  (Generalized  Reduced  Gradient  Method)  proposed  originally  by 
Abadie  [1]),  which,  to  the  author's  knowledge,  is  the  best  commercial 
code  for  solving  P.4.1. 

The  first  difficulty  in  designing  such  a test  is  to  determine  a 
measure  of  computational  efficiency.  Some  items  which  should  be  con- 
sidered -are  robustness,  "cost"  to  compute  a solution,  accuracy  of  the 
solution  with  respect  to  the  optimal  objective  value  and  with  respect 
to  tne  optimal  solution  vector,  and  ease  of  implementation.  Of  these, 
robustness  and  overall  expense  are  often  considered  to  be  the  most 
important.  Robustness  is  the  ability  to  handle  a wide  variety  of  problems 
under  various  starting  conditions.  For  this  reason  seventeen  distinct 
problems  are  presented  and  solved,  sometimes  from  different  starting 
points. 
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Haviiig  settled  on  costs  as  one  of  the  measures  of  efficiency 
the  next  task  is  to  determine  how  it  is  to  be  calculated.  One  possi- 
bility is  to  define  costs  as  tue  number  of  function  evaluations.  The 
reasoning  is  that  verj'  often  this  wili.  be  the  most  exjjensive  operation. 
There  are  several  drawbacks  to  using  this  statistic  as  the  sole  measure 
of  costs.  One  problem  is  that  it  totally  ignores  the  amount  of  work 
performed  between  function  evaluations.  Also,  function  evaluations  in 
one  method  may  be  quite  different  from  those  of  another  method.  For 
exajiiple,  the  fixed  point  code  will  require  a simplex  pivot  in  after 

each  gradient  evaluation,  but  it  will  never  evaluate  the  objective 
function.  Furthenaore,  due  to  some  of  the  techniques  of  Chapter  6, 
not  all  gradient  evaluations  will  require  a matrix  inverse.  On  the  other 
hand,  with  GRG,  the  objective  function  is  evaluated  many  times  but  no 
pivots  need  to  be  performed.  All  these  different  statistics  are  reported 
in  the  ensuing  tables;  however,  the  conclusion  is  that  it  is  necessary 
to  let  the  computer  determine  the  total  costs  via  the  CPU  time. 

Even  this  is  very  unsatisfactory  in  that  the  way  the  algorithms 
are  programmed,  the  access  of  data  and  the  amount  of  printing  can 
radically  affect  the  CPU  time.  Several  precautions  were  taken  to  reduce 
some  of  these  variances.  To  begin  with,  both  codes  were  run  on  the 
same  computer  (IBM  370/l6b)  and  on  identical  partitions  of  core.  Both 
were  compiled  under  Fortran  H,  0PT2  and  CPU  time  was  measured  from  the 
very  first  executable  statement  to  the  very  last.  The  fixed  point  code 
was  programmed  by  the  author  and  the  GRG  code  was  obtained  from  Leon 
Lasdon  and  Arvind  Jain  through  the  Systems  Optimization  Laboratory  at 
Stanford  tJniversity. 


The  final  step  was  to  find  seventeen  test  problems.  One  of  the 
major  difficulties  here  vras  that  many  of  the  problems  in  the  literature 
had  only  inequality  constraints.  Any  problem  of  this  kind  was  discarded 
immediately  since  the  entire  objective  was  to  see  how  this  algorithm 
would  perform  under  equality  constraints.  Since  problems  of  this  nature 
are,  in  general,  very  difficult  to  come  by  they  will  be  printed  here 
along  with  their  soui'ces,  knomi  solutions,  and  various  starting  points  in 
the  hopes  that  other  researchers  will  be  able  to  use  them. 


7.2.  Tables  of  Results  and  Conclusions . 

This  section  presents  the  results  of  the  fixed  point  code  and 
GRG  on  the  seventeen  problems  in  Section  7.3.  Table  1 gives  a summary 
of  the  characteristics  of  the  problems  and  Tables  2 and  3 give  a summary 
of  the  fixed  point  and  GRG  methods  respectively.  In  Table  2,  the  letter 
following  a number  refers  to  a different  starting  point  as  shown  in  the 
problem  description.  Also  in  Table  2,  the  number  of  gradient  calls 
requiring  a matrix  inverse  is  reported  whereas  Table  3 reports  all 
gradient  calls.  GRG  appears  to  be  som.ewhat  faster  in  solving  problems 
1,  7,  9,  12,  13  and  15,  whereas  the  fixed  point  code  seems  slightly 
faster  on  problems  2,  3,  5,  6,  8.  10,  11  and  li*.  At  first  gleuice 

there  is  reason  to  suspect  that  the  dimension  of  the  problem  is  a key 
factor.  A possible  explanation  for  this  is  that  as  the  dimension  goes 
up,  the  number  of  simplexes  traversed  by  the  fixed  point  algorithm  goes 
up  greatly.  In  order  to  test  this  possibility  a forty  dimensional  problem 
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was  invented  (Problem  l6)  and  GRG  proved  to  be  approximately  eleven  times 
faster  in  solving  it.  Note  that  this  problem  has  no  inequality  constraints. 
Since  inequality  constraints  add  to  the  complexity,  Problem  l6  was 
modified  by  adding  upper  bounds.  This  formed  Problem  1?.  An  intere.sting 
result  was  that  the  fixed  point  code  required  almost  twice  as  many  pivots 
(and  consequently  almost  twice  as  much  time)  to  solve  Problem  17  as  it 
did  to  solve  Problem  l6.  GRG,  on  the  other  hand,  actually  took  less 
time  to  solve  Problem  17  than  it  did  to  solve  Problem  16.  This  opens 
up  another  area  for  future  investigation. 

Nonetheless,  the  conclusion  from  these  tests  is  that  when  the 
dimension  of  the  original  problem  is  reduced  sufficiently  by  the  equality 
constraints,  the  fixed  point  approach  appears  to  be  more  effective  than 
GRG. 
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TABLE  1. 

PROBLEM  DESCRir-riON 


Problem 

Number 

Number  of 
Variables 

Number  of 
Eauality 
Constraints 



Lower 

Bounds 

Upper 

Bovmds 

— 

Nonlinearity  of 
Ob.jective 
Function 

Nonlinearity 

of 

Constraints 

1 

<£ 

] 

None 

None 

Quadratic 

Quadratic 

2 

1 

None 

None 

Mii.d 

Mild 

5 

3 

'J 

None 

None 

Strong 

Quadratic 

3 

2 

None 

None 

Cubic 

Quadratic 

5 

3 

2 

None 

None 

Strong 

Strong 

t. 

h 

p 

None 

None 

Strong 

Strong 

7 

5 

O 

<1 

None 

None 

Strong 

Strong 

8 

C 

2 

None 

None 

Strong 

Strong 

m 

5 

2 

All 

All 

Mild 

Mild 

10 

5 

None 

None 

Strong 

Strong 

11 

None 

None 

Strong 

Strong 

12 

6 

u 

All 

All 

Strong 

1^ 

10 

All 

1 

None 

strong 

Linear 

14 

11 

6 

All 

None 

Strong 

Linear 

15 

16 

8 

All 

All 

Qiiaciratic 

Linear 

l6 

4o 

20 

None 

None 

Quadratic 

Mild 

17 

ho 

20 

All 

All 

Quadratic 

Mild 

TABLE  2 


FIXED  POniT  RESULTS 


Problem 

Objective 

Number  of 

N+imber  of 

Number 

Value 

Newton 

Newton 

« 

Obtained 

Calls 

Iterations 

1 

1.39'* 

2lt 

1 

2 

.000005 

1+1 

1+0 

3(a) 

961.717 

20 

59 

3(b) 

961.717 

20 

59 

1* 

117.062 

16 

23 

5 

.1655 

35 

19 

6 

. 1+969 

52 

106 

7 

-2500.57 

1+0 

60 

8 

- 210.02'+ 

86 

132 

9 

-30665.5 

109 

71 

10(a) 

.05595 

28 

38 

10(b) 

.05395 

26 

50 

11(a) 

.02932 

liO 

39 

11(b) 

.02932 

37 

27 

11(c) 

27.552 

Ul* 

56 

11(d) 

27.552 

1+1+ 

56 

11(e) 

27.552 

Uk 

56 

12 

8827.6 

58 

71 

13 

-1*7.760 

1 

ll* 

1 

1 

15 

2hh.9 

1 

1 

16 

.7281+2 

1+27 

390 

17 

.73027 

785 

717 

♦ 

The  letters  In  parentheses  refer  to  the  dlfferenl 
**Only  those  gradient  calls  requiring  a matrix  lnv« 
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• ' 1 

Number  of 
Gradient 
Calls** 

Number  of 
Pivots 
Required 

CPU 

Time 

(in  secs) 

6 

2l* 

.07997 

9 

1*2 

.11625 

9 

26 

.0951+7 

9 

26 

.0951+7 

16 

16 

.08295 

33 

55 

. 1091+6 

29 

63 

.14749 

1+0 

1*1* 

. 13192 

26 

107 

.20190 

3'* 

165 

.20931 

9 

29 

. 11132 

6 

28 

.10978 

2h 

1*1* 

.13036 

21 

38 

.12151 

28 

1*9 

.13700 

28 

49 

.13700 

28 

49 

.13700 

1+0 

67 

.251*25 

1 

385 

. 56465 

1 

115 

.28489 

1 

967 

1.99 

265 

577 

16.01778 

785 

1084 

28.0649 

t starting  points. 


are  reported. 


Problem 

Ruaiber 

* 

Objective 

Value 

Obtained 

Number  of 
Newton 
Calls 

m 

1.59336 

13 

2 

0.00000 

1*7 

3(a) 

961.715 

30 

5(b) 

961.  a5 

1)0 

h 

117.056 

28 

5 

.1655 

13 

6 

-i4.U969 

7 

-2500.63 

29 

8 

-210.1*08 

75 

9 

-30655.5 

7 

10(a) 

.0539W 

2I+ 

10(b) 

— 

11(a) 

.02931 

51* 

BQI 

.02931 

29 

11(c) 

1*1*.  022 

1*1* 

11(d) 

27.872 

1*8 

11(e) 

607.017 

76 

1? 

8827.6 

7 

13 

-1*7.760 

9U 

Ih 

— 

15 

2l*l*.9 

156 

16 

.72839 

56 

17 

.73017 

3 

The  letters  In  parentheses  refer 


ORG  failed  to  obtain  correct 


TABLE  3 


ORG  RESULTS 


Number  of 
Newton 
Iterations 

Number  of 
Gradient 
Calls 

Number  of 
Objective 
Function  Calls 

CPU 

Time 

(in  secs) 

7 

4 

25 

.0715 

36 

9 

154 

.1139 

61 

9 

101 

. 11429 

86 

10 

135 

.12521 

21 

6 

50 

.09472 

3'7 

4 

51  1 

1 

.07924 

126  i 

14 

179 

.16045 

3i* 

8 

65 

.1284 

175 

15 

255 

.2036 

7 

6 

16 

I 

.0874 

42 

8 

68  1 

.U519 

— 

1 

1 

1 

1 

1 

76 

10 

113 

.13434 

51 

9 

84 

.12562 

30 

10 

78 

.13835 

111 

11 

160 

.15617 

152 

15 

239 

.20074 

5 

5 

14 

.08870 

1 

20 

101 

.32137 

~ 

1 

1 

— 

1 

1 

1 

37 

162 

.92266 

73 

150 

11 

i 

1.4873 

13 

5 

.4072 

to  the  different  starting  points, 
itlon  (could  be  user  error) . 
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7.3.  The  Test  Problems  . 

Since  equality  constrained  optimization  problems  are,  in  general, 
difficult  to  find  in  the  literature,  the  seventeen  problems  which  were 
used  for  the  comparative  tests  are  documented  here  along  with  their 
sources,  known  solutions  and  suggested  starting  points. 

Prob'.em  1. 

min  P(x^.x^)  = (x^-2)‘’  + (x,,-l)' 
s.t.  X,  -2x  +1=0 

1 c; 

2 

^1 

-r  + x; , - 1 < 0 

Optimal  Solution  and  Optimal  Objective  Value. 

X'  = (.825,  .911) 

P(x')  = 1.393 

Sxiggested  Starting  Point. 

X = (2,2^ 

Source . Himmelblau  [26]. 
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Problem 


min  P(x^,x.^,x^)  = (x^-Xg)  " + (x 
s.t.  x^  + x^Xg  + x^^^  - 5 = 0 

Optimal  Solution  and  Optimal  Ob.jectlve  Value. 

= (1,1,1) 

P(x' ) - 0 

Suggested  Starting  Point. 

X = (2.1,  .1,,  0) 

Source.  Avriel  ''2]. 

Problem  3. 


min 

P(x^, 

= 1000  - X^  - 

2x;  - x| 

s.t. 

2 

Xi  + 

2.2  ; 

- 23  = 0 

ox^  + 

llx^  + 

7x_, 

I 

II 

0 

Optimal  Solution  and  Optimal  Ob.iective  Value. 

X’  ^ (3. ‘>1212,  .216988,  3. 51/^7) 
P(x')  = 961.713 

Suggested  Starting  Points. 

(a)  X = (10,  10,  10) 

(b)  X = (-3,  -10,  3) 


Source.  Margaret  Wright  (33). 


Problem  4. 


min  P(x^,X2,x^)  = -(x^  + x^  + x^  - 7)^ 
s.t.  x'^  + Xg  + x|  - 2 = 0 
x^  - exp(x^)  = 0 

Optimal  Solution  and  Optimal  Objective  Value. 

X’  = (.17^X)6,  1.1961,  .7350) 

P(x')  = 117.062 

Suggested  Starting  Point. 

X = (0,1,1) 

Source.  Richard  Asmuth — Private  Communication. 

Problem  3. 

min  P{x^,x^,x^)  = expCx^x^  - x^) 

2 U 

s.t.  x^  + x^  - 2 = 0 

^1^2  “ ^ ^3  ~ ° 

Optimal  Solution  and  Optimal  Ob.jective  Value  . 

X'  =(-.68249ij,  .820716,  1.11204) 

P(x’)  = .1633052 

Suggested  StiJting  Point. 

X = (-1,1,1) 

Source.  Richard  McCord- -Private  Communication. 
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Problem  6. 


min  - -x^^Xj^  + (x^-1)  + {x^-x^)^+  (x^-1)^ 

s.t.  x^xj-  + sinCxj^  - x^)  - ij  = 0 

:■  2 h 

X2  ^ x^x^  -10  = 0 

Optimal  Solution  and  Optimal  Objective  Value. 

X'  = (2.03)036,  1,501623,  1.592i401,  1.1400879) 
p(x')  = -U. 496923 

Suggested  Starting  Point. 

X = (3.139,  3.162,  0,1) 

Source.  Richard  McCord — Private  Communication. 

Problem  7. 

min  P(x^,x2,x^,xj^,x^)  = 10x^X2^-6x^X2+X2X^+9  sin(x^-x^)  + x^x^x| 

^ ? 2 2‘  2 2 
s.t.  + ^2  ^^3  X|^  + x^  - 20  = 0 

2 

^1X3  + x^x^,  +2  =0 

r 2 

5 - x^x^^  - lOx^x^  < 0 

Optimal  Solution  and  Optimal  Objective  Value. 

X'  = (1.47963,  -2.63661,  I.O5I467,  -1.6ll'l,  2.67388) 

P(x')  = -2500.53 

S\;iggested  Starting  Point. 

X = (1.091,  -3.17*+,  1.214,  -l.6l4,  2.13I4) 

Source.  Modified  from  Himmelblau  [26]. 
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Problem  8. 


min  P(x^,x^,,x^,x^,x^)=-10x^xi^  - Gx^x^  + x^x^  + 9 sin(x^-x^)  + xj^x^x^ 


s.t.  x^  + x^  + Xj  + x^  + x^  - 20  = 0 


V4 

-X^Xj  - X^Xj^ 


- 5=0 

- 2 < 0 


Optimal  Solution  and  Optimal  Ob.iective  Value. 

X’  = (-.0814522,  3.69238,  2.48741,  .377134,  .173983) 

or 

X'  = (.0320,  3.6914,  ;'.4807,  .35993,  .29777) 

P(x')  = -210.024 

Suggested  Starting  Point. 

X = (1,1,1, 1,1) 

Source.  Modified  from  Himmelblau  [26]. 
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Problem  9. 


min  P(xj^,x^,x^.x|^,x^)  = 5.557^5't7x|  + ,8356891x^x^  + 37.P93239x^  - '^0792.141 

s.t.  83.334^407  + .0036858x^x  + .0006262x^X1  - .0022033x,x  - 92  = 0 

P J-  3 3 

9.300961  + .0047026x^x^  + .00l2547x^Xj  + .0019085x^Xj^  - 20  = 0 

80.51249  + .00713 17 x^jX^  + . 002995 5x^x,,  + .00021813x3  -100  < 0 

-80. 51240  - .0071317x,jX^  - .00299',  5x^X5  - .0002l8l3x^'  + 90  < 0 

78  < x^  < 102 
33  < < 45 

27  < x^  < 45 
27  < xi^  < 45 
27  < x^  < 45 

Optimal  Solution  and  Optimal  Ob.jectlve  Value. 

X'  = (7«,  33.  29.995,  36.776) 

P(x’)  = -30065.5 

Suggested  Starting  Point 

X = (78.62,  33.^44,  31.07,  44.18,  35.22) 

Source.  Modified  from  Colville  [ 8] . 
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Problem  10. 


min  P(x^.x^,x^,x^,x^)  = exp(x^x^x^xj^x^) 

P ?.  ’ 2 ■) 

s.t.  x^  + x^  + x^  + Xj^  + x^  - 10  = 0 

x^Xj  - 5xj^x^  =-  0 

3 3 

+1  = 0 


Optimal  Solution  and  Optimal  Objective  Value. 

X'  = (-1.71Y1U,  1.59571,  I.B2725,  -.763645,  -.765645) 

P(x')  = .0559499 


Suggested  Starting  Points. 

(a)  X = (-2,2, 2,-1, -1) 

(b)  X = (-1,-1,-!, -1,-1) 


Source.  Powell  [42)  also  Margaret  Wright  [55]. 
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Problem  11. 


min  P(x^,X2,x^,Xi^,x^)  = (x^-1)^  + (x^-x^)^  + + (xi^-x  )' 


s.  t. 


x^  + X2  + x^-P-3y/2=0 
X^  - x|  + Xj^  + 2 - 2 y/p  ^ 0 


x^x, 


-2  = 0 


Optimal  Solution  and  Optimal  Objective  Values. 

X'  = (1.11663,  1.220'+4,  1.33779,  1.97277,  1.79110) 

P(x’)  = .02931B 

X'  = (-2.79^^7,  3.004i4,  .205371,  3.87^74,  -.716623) 
P(x')  = 60^036 

X'  = (-1.2/305,  2.41035,  1.19486,  -.154259,  -1.57103) 
P(x’ ) = 27.8719 

X'  = (-.703393,  2.63570,  -.0963618,  -1.79799,  -2.mJ,336) 
p(x')  = 44.021. 


Suggested  Starting  Points. 

(a)  X = (1,1, 1,1.1) 

(b)  X = (2, 2. 2, 2. 2) 

(c)  X = (-1,3, -1/2, -2, -3) 

(d)  X = (-1,2, 1,-2, -2) 

(e)  X = (-2, -2, -2, -2, -2) 


Source.  Miele,  Moseley,  Levy  and  Coggins  [6];  also  Margaret  Weight  [55]. 
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ProWem  1?. 


min 


s « • 


P(x^,X^3,X^,Xj^,X,^,Xg)  = P’(x^)  + 

o 

x^  - c + (x^Xj^  cos(t)-Xg)  - x^  A 

x^  + (x^X)^  cos(b+x^)  - Xj^  A 

2 

(xjXj^  sin(b-Xg)  - x^  A 
x^  + (x^x^  sin(t)+Xg)  - x^  A 


cos (b-a) ) /b 

cos(b-a) ) /B 

sin(b-a) ) /B 
sin(b-a) ) /B 


= 0 

= 0 

D = 0 
= 0 


0 < x^  < Uoo 
0 < < 1000 
JUO  < Xj  < il20 
3k)  < X|^  < it:’0 

-1000  < x^  < 1000 

0 < Xg  < . 3236 

and 


bP'  f . 


and 


(Xg) 


where  A = .90798 

B = 131.078 
a = .00889 

b = 1.481+77 

c = 300 
D = 200 

30  if  0 < x^  < 300 

31  if  300  < x^  < kO 


28 

if 

0 

< 

X.,  < 100 

* 

2 

29 

if 

100 

< 

x^  < 200 

2 

30 

if 

200 

< 

Xg  < 500 

Optimal  Solution  and  Optimal  Ob.lective  Value. 

X'  = (107.834,  196.295,  373.836,  420.002,  21.293,  .15327) 
P(x')  = 8827.595 
Suggested  Starting  Point. 

X = (390,  1000,  419.5,  3k. 5,  198.175,  .5) 

Source . Coleville  [8]. 


Problem  1^. 


min  P(x^, 


10  10 
y,  X.  (c.  + ^n(x.  / '1  X . ) ) 
i=l  .>1  ^ 


+ 

2x^  + 

2x,  + X,  + 
5 0 

*10 

-2  = 0 

+ 

2Xr.  + 

X + 

0 

li 

1 

5 

0 

7 

+ 

X..,  + 

Xq  + 2x^  + 

X,  ^ 

-1=0 

f 

8 9 

10 

where 

c = (-6.089,  -I7.l6i+,  -l>k.03h,  -5.9lh,  -24.721,.  -14.986, 
-24.1,  -10.?’08,  -26,662,  -22.179) 


Optimal  Solution  and  Optimal  Ob.jective  Value. 

X’  = (.0406,  .147^  .7852,  .0014,  .)i855,  .0007, 
.0274,  .01%,  .0:^75,  .0969) 

P(x')  = -47.761 


Suggested  Starting  Point. 

x = (.1,  ...  , .1) 


Source.  Himmelblau  [26], 


111 


Problem  1^. 


min  P(cq,  ...  , Cg,  k^, 


5.t.  kj^  = kg  = 100 


i+1 


= 1.3k^  - , 


where 


V = 4.3026 


5 T 

Itr)  = - ^ ”''^/(l-V) 

i=0  ^ 


i = 0,...,  3 


Optimal  Solution  and  Optimal  Ob.lectlve  Value. 

(c',k')  = (28.474,  29.282,  30.139,  31.047,  32.010,  35.031, 
101.526,  102.702,  103.37^,  103.3^9,  102.530) 

P(c',k’)  = .000120804 

Suggested  Starting  Point. 

(c,k)  = (101,  101,  ...  , 101) 

Source.  Professor  Alan  Manne — Private  Communication. 
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Problem  I'y 


min 


s.t. 


where 


2 ? 

P(x  , = E 7!  a (x  + X + l)ix  + X + 1) 

i=l  ^ J J 

l6 

E b X = c,  , i = 1, . . . ,a 

j=l 


0 < X.  !; 

“ o 


j = 1, . . . , l6 


A = 

1 

2 

3 

5 

6 

7 

8 

9 

10 

11 

12 

13 

l4 

15 

ra 

1 

1 

1 

1 
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fl 

B 
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B 
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fll 
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1 
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fl 
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■ 

B 

B 

B 

B 

B 

B 

B 
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6 

■ 

■ 

fl 

B 

B 

B 
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B 

B 
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■ 

■ 

B 

B 

B 

B 

B 

B 

■ 

B 

9 

■ 

1 

B 

B 

1 

B 

B 

10 

1 

■1 

B 

11 

■ 

fl 

B 

fll 

B 

1 

■ 

12 

B 

B 

m 

Bl 

B 

B 

■1 

13 

■ 

B 

■ 

B 

B 

1 

B 

14 

□ 

■ 

B 

B 

■ 

B 

B 

15 

B 

■ 

B 

B 

B 

B 

B 

16 

□ 

■ 

B 

B 

Bi 

B 

B 

B 

B 

c = 

(p. 

5,  1 

.1, 

-3.1 

-3 

5,  1.3, 

2.1, 

2.3, 

-1.5)  and 
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Problem  l6. 

1 20  I4O 

min  P(x^, . . =-  Z x.  + Z x. 

1=1  ^ j=21 

20 

s.t.  exp(-  Z X ) - 1 = 0,  1 < i < ;^0 

J=1 


Optimal  Solution  and  Optimal  Ob.iectlve  Value. 


x;  = 


-.22J+38  if  1 < i < 20 

.011251*  if  21  < i < 1+0 


P(x')  = .728399 


Siiggested  Starting  Point. 

(0  if  1 < i < 20 

X.  = j 

^ I 1 if  21  < i < 1+0 


Source . Proposed  by  author. 
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Problem  17. 


1 '0  ^0 

min  P(x^,.,.,Xj^q)  = - F.  x"  + Z x. 

i=l  ^ j=21  ^ 


20 

s.t.  exp(-  Z X ) - 1 = 0 1 £ i < 20 

j=l  ^ 


^i+20 


< .01 


1 < i < 20 


Optimal  Solution  and  Optimal  Objective  Value. 

-.23025  if  1 < i < 20 

^i  = 

.01  if  21  < i < 40 


Suggested  Starting  Point. 

X.  = 0 
1 


1 < i < 40 


Source.  Proposed  by  author. 
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Al^PENDIX  A 


With  the  theory  of  Chapters  1 and  )(  it  is  now  possible  to  show 
that  P.I4.I  is  in  fact  a special  case  of  decomposability  of  a point  to 
set  map.  This  is  formally  stated  and  proved  in 

Theorem.  Suppose  A.'Kl-A.i4.5  hold  and  that  in  addition  P,  G,  h and  Q 

are  all  differentiable  on  their  respective  domains.  Define  the  function 

L:R  R by  L(x)  = V P(x,h(x))  + V P(x,h(x))'^  Dh(x).  Then  the  point 

^ y 

to  set  map  S:R™  x R*^  ->  (r”  x r”)  defined  by 

(x,y)  - { (L(x),G(x,y)))  if  t(x)  <0 

S(x,y)  = • (x,y)-hull  (at(x)  U {L(x) ) )x{G(x,y) ) if  t(x)  =0 

. (x,y)  - dt(x)  X {G(x,y))  if  t(x)  >0 

satisfies  (S,r'''  X R*^)  is  decomposable. 

Proof.  Let  X = R*",  Y = r",  Z = X x Y.  Define  S^:Z  ->  (X)*  by 

if  t(x)  < 0 

if  t(x)  = 0 
if  t(x)  > 0 

117 


■ X - {L(x)} 

S^(x,y)  = ' x-hull(St(x)  U {L(x)]) 

■ x - f^t(x) 

and  S ;Z  -»  (Y)*  by 

O 


Sg(x,y)  = y - {G(x,y)]  . 

Property  (l)  of  D.2,2  holds  by  construction.  Furthermore,  property  (2) 
of  D.2.2  holds  trivially  since  for  each  x G r'"  , 

Sg(x,h(x))  = h(x)  - {G(x,h(x)))  = h(x)  - (0)  = {h(x)}.  || 

It  is  of  course  important  to  note  that  under  the  conditions  developed  in 

solution  to  P.1+,1  may  be  obtained  by  finding  a fixed  point  of 
set  map  S^:X  -»  (X)  defined  by  S^(x)  = S^(x,h(x)). 


Chapter  1+,  a 
the  point  to 
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In  the  past  decade,  several  constructive  proofs  of  l-he  Brouwer  and  Kakutanl 
fixed  point  theorems  have  emerged.  These  proofs  have  been  developed  into 
algorithms  (known  in  ti>e  literature  as  complementary  pivot  algoritlims)  which 
search  for  fixed  point.s  on  unbounded  regions,  in  turn  these  algoritluns  have 
been  used  to  solve  problems  arising  in  economics,  engd^u'ering  and  other 
branches  of  applied  mathematics.  An  important  applicat/lisn  for  whicli  this 
method  was  cumbersome  and  inefficient  to  use  was  that  of  o^vtimi zing  an  objec- 
tive function  subject  to  both  equality  and  inequali  t,y  constrbvuits  (hereafter 
referred  to  as  the  general  constraiiied  optimization  prob  1 em ) . ^ One  result  of  -t 
d iguop-tu ti oi>  is  tiie  most  efficient  complementary  pivot  algorithm  to  date 
for  handling^Ww^problem.  The  second  major  contribution  ef  thio  thesi'^  i;i 
a general  structure  on  fixed  point  problems  which,  whei>  present,  enables  one 
to  work  in  a lower  dimensional  space.  It  is  shown  that  the  general  constrained 
optimization  problem  may  sometimes  be  formulated  as  a fixed  point  jiroblem 
i possessing  this  property. 

I 

The  basic  approach  adopted  in  this  work  for  handling  the  general  con- 
strained optimization  problem  is  to  use  an  implicit  function  (derived  from 
the  equality  constraints)  to  solve  for  some  dependent  variables  in  terms  of 
the  remaining  independent  ones.  Under  certain  circumstances,  a fixed  point 
algorithm  may  be  used  to  search  for  optimal  values  of  the  independent,  vari- 
iables  while  Nc'wton's  method  is  used  to  determine  values  of  the  dependent 
[variables.  Theoretical  conditions  on  the  original  functions  are  developed 
to  guarantee  that  the  fixed  point  algoritiim  converges  to  a solution  and  vari- 
I ous  techniques  are  devised  to  enhance  the  overall  efficiency.  _ 


To  help  ascertain  the  value  of  this  method,  comparative  computer  tests 
are  run  against  the  Generalized  Reduced  Gradient  (GRG)  algorittim  which  is  a 
well  established  nonlinear  programming  code.  This  method  was  selected  as 
the  basis  for  comparison  because,  to  t.he  author's  knowledge,  it  is  the  best 
commercial  code  for  solving  the  general  constrained  optimization  problem. 
Seventeen  test  xiroblems  were  taken  from  various  sources.  The  fixed  point 
code  solved  all  seventeen  and  GRG  solved  sixt.een.  This  supports  the  robust- 
ness of  the  fixed  point  approach.  As  to  the  computer  times,  the  fixed  point 
code  proved  to  be  as  fast  or  faster  than  GRG  on  the  lower  dimernuonal  problems. 
As  the  dimension  increased,  however,  the  trend  reversed  and  on  a forty  dimen- 
sional problem  GRG  was  apj’roximately  eleven  times  faster.  The  conclusion 
is  that  when  the  dimension  of  the  original  problem  can  be  sufficiently  reduced 
by  the  equality  constraints,  the  fixed  point  approach  appears  to  be  more 
effective. 
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